通信与电子信息专业英语 课件 Unit 6 Digital Image Processing

Unit6DigitalImageProcessing6.1ComponentsofanImageProcessingSystem

6.2ExamplesofFieldsThatUseDigitalImageProcessing

6.3IntroductiontoPatternRecognition6.1ComponentsofAnImageProcessingSystemWordsandExpressionsBackgroundsText4peripheraldevice 外部设备；外围设备graphicsprocessingunits(GPU) 图形处理单元Designedforparallelprocessing,theGPUisusedinawiderangeofapplications,includinggraphicsandvideorendering.Althoughthey’rebestknownfortheircapabilitiesingaming,GPUsarebecomingmorepopularforuseincreativeproductionandartificialintelligence(AI).deepconvolutionalnetwork 深度卷积网络WordsandExpressionsTypicalCNNarchitecture5词汇拓展TPU:TensorProcessingUnit，an

AIaccelerator

application-specificintegratedcircuit

(ASIC)developedby

Google

for

neuralnetwork

machinelearning,usingGoogle'sown

TensorFlow

software.DPU:DataProcessingUnit,a

channelcontroller,aprogrammablespecialized

electroniccircuit

with

hardwareacceleration

of

dataprocessing

for

data-centriccomputing. WordsandExpressionsTensorProcessingUnit3.0designedbyGooglein2016WordsandExpressionssensor n.传感器digitizer n.数字化仪；数字化器arithmeticlogicunit(ALU)算术逻辑单元6WordsandExpressionsframe n.帧buffer n.缓存器；缓冲器；缓存区magnetic adj.磁的，有磁性的stereo adj.立体的；立体声的7Diagramdescribingrelationshipofimagedisplacementtodepthwithstereoscopicimages,assumingflatco-planarimagesComputerstereovision

istheextractionof3Dinformationfromdigitalimages,suchasthoseobtainedbya

CCDcamera.

WordsandExpressionsresolution n.分辨率projection n.投影bandwidth n.带宽8Anillustrationofhowthesameimagemightappearatdifferentpixelresolutions,ifthepixelswerepoorlyrenderedassharpsquares.CameraprojectionmodelBackgroundsDigitalImageProcessing,4thedition,byR.C.andWoods,R.E9TextbookaboutDIPBackgrounds10StanfordEE368/CS232:DigitalImageProcessing/class/ee368/index.htmlTextOutlineComponentsofanimageprocessingsystemThedevelopmentofdigitalimageprocessingComponentsofimageprocessingsystem11Componentsofageneral-purposeimageprocessingsystem.ThedevelopmentofdigitalimageprocessingInthelate1990sandearly2000s,anewclassofadd-onboards,calledgraphicsprocessingunits(GPU)wereintroducedforworkon3Dapplications,suchasgamesandother3Dgraphicsapplications.在20世纪90年代末和21世纪初，一种称为图形处理单元（GPU）的新型附加板被引入3D应用，如游戏和其他3D图形应用。12ThedevelopmentofdigitalimageprocessingItwasnotlongbeforeGPUsfoundtheirwayintoimageprocessingapplicationsinvolvinglarge-scalematriximplementations,suchastrainingdeepconvolutionalnetworks.没过多久，GPU就进入了涉及大规模矩阵实现的图像处理应用，例如训练深度卷积网络。13AMD2022-2024GPURoadmapThedevelopmentofdigitalimageprocessingInadditiontoloweringcosts,themarketshift

fromsubstantialperipheraldevicestoadd-onprocessingboardsalsoservedasacatalystforasignificantnumberofnewcompaniesspecializinginthedevelopmentofsoftwarewrittenspecificallyforimageprocessing.除了降低成本外，市场从大量外围设备向附加处理板的转变也催生出大量专门开发专用于图像处理软件的新公司。14ComponentsofanimageprocessingsystemSpecializedimageprocessinghardwareusuallyconsistsofthedigitizerjustmentioned,plushardwarethatperformsotherprimitiveoperations,suchasanarithmeticlogicunit(ALU),thatperformsarithmeticandlogicaloperationsinparallelonentireimages.专门的图像处理硬件通常包括刚才提到的数字化仪，以及执行其他基本操作的硬件，例如算术逻辑单元（ALU），它在整个图像上并行执行算术和逻辑操作。15ComponentsofanimageprocessingsystemInotherwords,thisunitperformsfunctionsthatrequirefastdatathroughputs(e.g.,digitizingandaveragingvideoimagesat30frames/s)thatthetypicalmaincomputercannothandle.OneormoreGPUs(seeabove)alsoarecommoninimageprocessingsystemsthatperformintensivematrixoperations.换句话说，该单元执行的功能需要快速的数据吞吐量（例如，以30帧/秒的速度对视频图像进行数字化和平均化），而典型的主计算机无法处理这些功能。一个或多个GPU（见上文）在执行密集矩阵运算的图像处理系统中也是常见的。16ComponentsofanimageprocessingsystemWhendealingwithimagedatabasesthatcontainthousands,orevenmillions,ofimages,providingadequatestorageinanimageprocessingsystemcanbeachallenge.在处理包含数千甚至数百万图像的图像数据库时，在图像处理系统中提供足够的存储空间可能是一个挑战。17ComponentsofanimageprocessingsystemInsomecases,itisnecessarytohavestereodisplays,andtheseareimplementedintheformofheadgearcontainingtwosmalldisplaysembeddedingoggleswornbytheuser.在某些情况下，需要有立体显示器，这些显示器以头盔的形式实现，其中包含嵌入用户佩戴的护目镜中的两个小显示器。18Becauseofthelargeamountofdatainherentinimageprocessingapplications,thekeyconsiderationinimagetransmissionisbandwidth.由于图像处理应用中固有的大量数据，图像传输的关键考虑因素是带宽。19Componentsofanimageprocessingsystem6.2ExamplesofFieldsThatUseDigitalImageProcessingWordsandExpressionsBackgroundsTextWordsandExpressionsinfrared adj.红外线的；使用红外线的

n.红外线；红外区acoustic adj.声音的，听觉的ultrasonic adj.超声的Approximatefrequencyrangescorrespondingtoultrasound,withroughguideofsomeapplicationsUltrasoundimage(sonogram)ofafetusinthewomb,viewedat12weeksofpregnancy(bidimensionalscan)WordsandExpressionselectronmicroscopy 电子显微术；电子显微镜学；电子显微法syntheticimage 合成的图像visualization n.可视化Syntheticimagesarecomputergeneratedimageswhichrepresenttherealworld.RealimageandSyntheticimageWordsandExpressionsDatavisualization 数据可视化Datavisualizationis

thepracticeoftranslatinginformationintoavisualcontext,suchasamaporgraph,tomakedataeasierforthehumanbraintounderstandandpullinsightsfrom.Themaingoalofdatavisualizationistomakeiteasiertoidentifypatterns,trendsandoutliersinlargedatasets..WordsandExpressionsradiation n.辐射spectrum n.频谱electromagneticwave 电磁波wavelength n.波长Electromagneticspectrum

withvisiblelighthighlightedWordsandExpressionsphoton n.光子spectral adj.谱的，光谱的imaging n.成像Medicalimaging

isthetechniqueandprocessof

imaging

theinteriorofabodyforclinicalanalysisandmedicalintervention,aswellasvisualrepresentationofthefunctionofsomeorgansortissues.Plainx-rayofthewristandhandOneframeofa

CTscan

ofthechestshowingtheheartandlungs.OneframeofanMRIscanoftheheadshowingtheeyesandbrain.WordsandExpressionsremotesensing 遥感microwave n.微波Microwave

isaformof

electromagneticradiation

with

wavelengths

rangingfromaboutonemetertoonemillimetercorrespondingto

frequencies

between300

MHzand300

GHzrespectively.IllustrationofremotesensingDeeplearninginremotesensingWordsandExpressionssnapshot n.快照lens n.透镜，镜片antenna n.天线magneticresonanceimaging(MRI) 磁共振成像Xband

marineradar

slotantenna

onship,8–12

GHz.Para-sagittal

MRIofthehead,with

aliasing

artifacts

(noseandforeheadappearatthebackofthehead)Backgrounds28NorthwesternUniversity数字图像和视频处理基础/learn/digitalTextOutline29ExamplesoffieldsthatusedigitalimageprocessingGamma-RayImagingX-RayImagingImagingintheUltravioletBandImagingintheVisibleandInfraredBandsImagingintheMicrowaveBandImagingintheRadioWaveBandOtherImagingModalitiesTheelectromagneticspectrumarrangedaccordingtoenergyperphoton.ExamplesoffieldsthatusedigitalimageprocessingToday,thereisalmostnoareaoftechnicalendeavorthatisnotimpactedinsomewaybydigitalimageprocessing.Wecancoveronlyafewoftheseapplicationsinthecontextandspaceofthecurrent.如今，几乎没有任何一个技术领域不受数字图像处理的影响。在当前讨论的上下文和领域中，我们只能介绍其中的部分应用。30ExamplesoffieldsthatusedigitalimageprocessingOneofthesimplestwaystodevelopabasicunderstandingoftheextentofimageprocessingapplicationsistocategorizeimagesaccordingtotheirsource(e.g.,X-ray,visual,infrared,andsoon).要基本了解图像处理应用程序的范围，最简单的方法之一是根据图像的来源（例如，X射线、可见光、红外线等）对图像进行分类。31Real-timeMRIofa

humanheart

ataresolutionof50

msElectromagneticwavescanbeconceptualizedaspropagatingsinusoidalwavesofvaryingwavelengths,ortheycanbethoughtofasastreamofmasslessparticles,eachtravelinginawavelikepatternandmovingatthespeedoflight.电磁波可以被概念化为传播波长不同的正弦波，也可以被认为是无质量粒子流，每个粒子以波的形式传播，并以光速移动。32ExamplesoffieldsthatusedigitalimageprocessingTheinfraredbandoftenisusedinconjunctionwithvisualimaging,sowehavegroupedthevisibleandinfraredbandsinthissectionforthepurposeofillustration.红外波段通常与视觉成像结合使用，因此为了便于说明，我们在这部分中将可见光和红外波段合并为一组。33ExamplesoffieldsthatusedigitalimageprocessingVisiblelightandinfraredimagesExamplesoflightmicroscopyimages.(a)Taxol(anticanceragent),magnified250×.(b)Cholesterol—40×.(c)Microprocessor—60×.(d)Nickeloxidethinfilm—600×.(e)SurfaceofaudioCD—1750×.(f)Organicsuperconductor—450×.34Examplesoffieldsthatusedigitalimageprocessing(a)(b)(c)(d)(e)(f)Theuniquefeatureofimagingradarisitsabilitytocollectdataovervirtuallyanyregionatanytime,regardlessofweatherorambientlightingconditions.成像雷达的独特之处在于，它能够在任何时间收集几乎任何地区的数据，而不管天气或环境照明条件如何。35Examplesoffieldsthatusedigitalimageprocessing913×913satelliteimageofHurricaneAndrewRadarimageAnimagingradarworkslikeaflashcamerainthatitprovidesitsownillumination(microwavepulses)toilluminateanareaonthegroundandtakeasnapshotimage.Insteadofacameralens,aradarusesanantennaanddigitalcomputerprocessingtorecorditsimages.Inaradarimage,onecanseeonlythemicrowaveenergythatwasreflectedbacktowardtheradarantenna.成像雷达的工作原理就像闪光灯一样，它提供自己的照明（微波脉冲）来照亮地面上的区域并拍摄快照图像。雷达使用天线和数字计算机处理来记录图像，而不是相机镜头。在雷达图像中，人们只能看到反射回雷达天线的微波能量。36ExamplesoffieldsthatusedigitalimageprocessingAlthoughimagingintheelectromagneticspectrumisdominantbyfar,thereareanumberofotherimagingmodalitiesthatarealsoimportant,includingacousticimaging,electronmicroscopy,andsynthetic(computer-generated)imaging.尽管电磁频谱成像目前占主导地位，但还有许多其他成像方式也很重要，包括声学成像、电子显微镜成像和合成（计算机生成）成像。37Examplesoffieldsthatusedigitalimageprocessing6.3IntroductiontoPatternRecognitionWordsandExpressionsBackgroundsTextWordsandExpressionspatternrecognition模式识别Patternrecognitionistheabilityofmachinestoidentifypatternsindata,andthenusethosepatternstomakedecisionsorpredictions.spatial adj.空间的PatternRecognitiontosolvethecomputervisiontaskObjectDetection.Examplesofpatterns:Soundwave,treespecies,fingerprint,face,barcode,QR-code,handwriting,orcharacterimage.WordsandExpressionsfeatureextraction特征提取classification n.分类Theobjectstobeclassifiedarefirstsensedbyatransducer(camera),whosesignalsarepreprocessed,thenthefeaturesextractedandfinallytheclassificationemitted(hereeither“salmon”or“seabass”).WordsandExpressionsnoisereduction 降噪Noiseisdefinedasverygeneralterms:anypropertyofthesensedpatternduenottothetrueunderlyingmodelbutinsteadtorandomnessintheworldorthesensors.Allnon-trivialdecisionandpatternrecognitionproblemsinvolvenoiseinsomeform.Insomecasesitisduetothetransductioninthesignalandwemayconsigntoourpreprocessortheroleofcleaningupthesignal.WordsandExpressionsenhancement n.增强restoration n.复原segmentation n.分割Digitalphotographrestoration

and

colorization

using

artificialintelligenceImageenhancementImagesegmentationwithdeeplearningWordsandExpressionslabel n.标签prototype n.原型optimumstatisticalclassifier 最优统计分类器In

statistics,

classification

istheproblemofidentifyingwhichofasetof

categories

(sub-populations)an

observation

(orobservations)belongsto.

ImagelabellingWordsandExpressionsneuralnetwork神经网络Convolutionalneuralnetwork卷积神经网络rawdata 原始数据WordsandExpressionscharacterrecognition字符识别Simplifiedviewofa

feedforward

artificialneuralnetworkSceneTextRecognition(STR)forroadsignreadingWordsandExpressionsunlabeleddata 未标记的数据cluster n.群，组Theresultofaclusteranalysisshownasthecoloringofthesquaresintothreeclusters.k-meansseparatesdataintoVoronoicells,whichassumesequal-sizedclusters(notadequatehere)WordsandExpressionstrainingset训练集validationset验证集testset测试集In

machinelearning,acommontaskisthestudyandconstructionof

algorithms

thatcanlearnfromandmakepredictionson

data.Suchalgorithmsfunctionbymakingdata-drivenpredictionsordecisions,

throughbuildinga

mathematicalmodel

frominputdata.

Threedatasetsarecommonlyusedindifferentstagesofthecreationofthemodel:training,validationandtestsets.Themodelisinitiallyfitona

trainingdataset,

whichisasetofexamplesusedtofittheparameters.Successively,thefittedmodelisusedtopredicttheresponsesfortheobservationsinaseconddatasetcalledthe

validationdataset.Finally,the

testdataset

isadatasetusedtoprovideanunbiasedevaluationofa

final

modelfitonthetrainingdataset.WordsandExpressionsiteration n.迭代supervisedlearning 有监督学习Supervisedlearning(SL)

isa

machinelearning

paradigmforproblemswheretheavailabledataconsistsoflabelledexamples,meaningthateachdatapointcontainsfeatures(covariates)andanassociatedlabel.unsupervisedlearning 无监督学习Unsupervisedlearning

isa

machinelearning

paradigmforproblemswheretheavailabledataconsistsofunlabelledexamples,meaningthateachdatapointcontainsfeatures(covariates)only,withoutanassociatedlabel.

BackgroundsRelatedAreasofImageProcessing:DigitalImageProcessing:BerndGirod,©2013-2015StanfordUniversity--Introduction21BackgroundsPatternRecognitionForMachineVision/courses/9-913-pattern-recognition-for-machine-vision-fall-2004/PatternRecognitionAndAnalysis/courses/mas-622j-pattern-recognition-and-analysis-fall-2006/50BackgroundsMachineVision/courses/6-801-machine-vision-fall-2020/51TextIntroductiontopatternrecognitionIntroductiontopatternrecognitionMainstagesinpatternrecognitionBasicapproachesusedforimagepatternrecognition52Introductiontopatternrecognition53Imaginewhatmodernlifewouldbelikewithoutmachinesthatreadbarcodes,processbankchecks,inspectthequalityofmanufacturedproducts,readfingerprints,sortmail,andrecognizespeech.想象一下，如果没有读取条形码、处理银行支票、检查制成品质量、读取指纹、分拣邮件和识别语音的机器，现代生活会是什么样子。Introductiontopatternrecognition54Thatis,givenapatternorsetsofpatternswhoseclassisunknown,thejobofapatternrecognitionsystemistoassignaclasslabeltoeachofitsinputpatterns.也就是说，给定一个或多个类别未知的模式，模式识别系统的工作就是为其每个输入模式分配一个类别标签。55Therearefourmainstagesinvolvedinrecognition:(1)sensing,(2)preprocessing,(3)featureextraction,and(4)classification.Intermsofimageprocessing,sensingisconcernedwithgeneratingsignalsinaspatial(2D)orhigher-dimensionalformat.Preprocessingdealswithtechniquesfortaskssuchasnoisereduction,enhancement,restoration,andsegmentation.Classificationdealswithusingasetoffeaturesasthebasisforassigningclasslabelstounknowninputimagepatterns.识别涉及四个主要阶段：（1）感知，（2）预处理，（3）特征提取和（4）分类。在图像处理方面，感知是以空间（2D）或更高维格式生成信号。预处理包括了降噪、增强、恢复和分割等任务的技术。分类是指应用一组特征作为将类标签分配给未知输入图像模式的基础。Mainstagesinpatternrecognition56Inthefollowingsection,wewilldiscussthreebasicapproachesusedforimagepatternclassification:(1)classificationbasedonmatchingunknownpatternsagainstspecifiedprototypes,(2)optimumstatisticalclassifiers,and(3)neuralnetworks.在下面的部分中，我们将讨论用于图像模式分类的3种基本方法：（1）根据指定原型匹配未知模式的分类方法；（2）最佳统计分类方法；（3）神经网络方法。Basicapproachesusedforimagepatternrecognition57Labeleddata:Labeleddata

isagroupof

samples

thathavebeentaggedwithoneormorelabels.Labelingtypicallytakesasetofunlabeleddataandaugmentseachpieceofitwithinformativetags.Forexample,adatalabelmightindicatewhetheraphotocontainsahorseoracow,whichwordswereutteredinanaudiorecording,whattypeofactionisbeingperformedinavideo,whatthetopicofanewsarticleis,whattheoverallsentimentofatweetis,orwhetheradotinanX-rayisatumor.Basicapproachesusedforimagepatternrecognition58Labeldata:Basicapproachesusedforimagepatternrecognition59Unlabeleddata:Unlabeleddatais

adesignationforpiecesofdatathathavenotbeentaggedwithlabelsidentifyingcharacteristics,propertiesorclassifications.Unlabeleddataistypicallyusedinvariousformsofmachinelearning.Basicapproachesusedforimagepatternrecognition60Aclassicexampleoflabeleddataisthecharacterrecognitionproblem,inwhichasetofcharactersamplesiscollectedandtheidentityofeachcharacterisrecordedasalabelfromthegroup0through9andathroughz.有标签数据的一个典型示例是字符识别问题，在该问题中，收集一组字符样本，并将每个字符的标识记录为一组0到9及a到z的标签。Basicapproachesusedforimagepatternrecognition61Anexampleofunlabeleddataiswhenweareseekingclustersinadataset,withtheaimofutilizingtheresultingclustercentersasbeingprototypesofthepatternclassescontainedinthedata.无标签数据的一个例子是，当我们在某数据集中寻找聚类时，目标是利用由此产生的聚类中心作为数据中包含的模式类的原型。BasicapproachesusedforimagepatternrecognitionBasicapproachesusedforimagepatternrecognition62Whenworkingwithalabeleddata,agivendatasetgenerallyissubdividedintothreesubsets:atrainingset,avalidationset,andatestset(atypicalsubdivisionmightbe50%training,and25%eachforthevalidationandtestsets).当使用有标签数据时，给定的数据集通常被细分为三个子集：训练集、验证集和测试集（典型的细分可能是50%的训练，验证集和试验集各25%）。Basicapproachesusedforimagepatternrecognition63Inthismode,aclassifierisgiventheclasslabelofeachpattern,theobjectivebeingtomakeadjustmentsintheparametersiftheclassifiermakesamistakeinidentifyingtheclassofthegivenpattern.在这种模式下，分类器得到每个模式的类别标签，目的是如果分类器在识别给定模式的类型时出错，则对参数进行调整。64Iftraining/validationresultsareacceptable,buttestresultsarenot,wesaythattraining/validation“overfit”thesystemparameterstotheavailabledata,inwhichcasefurtherworkonthesystemarchitectureisrequired.如果训练/验证结果是可接受的，但测试结果不是，我们说训练/验证使系统参数“过拟合”了可用数据，在这种情况下，需要对系统架构进行进一步的工作。Basicapproachesusedforimagepatternrecognition65Overfitting:In

mathematicalmodeling,

overfitting

is"theproductionofananalysisthatcorrespondstoocloselyorexactlytoaparticularsetofdata,andmaythereforefailtofittoadditionaldataorpredictfutureobservationsreliably".

An

overfittedmodel

isa

mathematicalmodel

thatcontainsmore

parameters

thancanbejustifiedbythedata.

Theessenceofoverfittingistohaveunknowinglyextractedsomeoftheresidualvariation(i.e.,the

noise)asifthatvariationrepresentedunderlyingmodelstructure.BasicapproachesusedforimagepatternrecognitionThegreenlinerepresentsanoverfittedmodelandtheblacklinerepresentsaregularizedmodel.Whilethegreenlinebestfollowsthetrainingdata,itistoodependentonthatdataanditislikelytohaveahighererrorrateonnewunseendata,comparedtotheblacklin