机器视觉综述课件

ComputerVision

(机器视觉)ImagebyComputerVision(机器视觉)ImagebyToday’sTalkWhatisComputerVision?WhyStudyComputerVision?HowVisionisUsedNow?OverviewofComputerVisionAlgorithmChallengesofComputerVisionQuestions2Today’sTalkWhatisComputerVWhatiscomputervision?Terminator23Terminator5Whatiscomputervision?TerminEverypicturetellsastory4GoalofcomputervisionistowritecomputerprogramsthatcaninterpretimagesEverypicturetellsastory4GoCancomputersmatch(orbeat)humanvision?5Cancomputersmatch(orbeat)WhatisComputerVision?AutomaticunderstandingofimagesandvideoComputingpropertiesofthe3Dworldfromvisualdata(measurement)

6WhatisComputerVision?Automa

1.VisionformeasurementReal-timestereoStructurefrommotionNASAMarsRoverPollefeysetal.Multi-viewstereofor

communityphotocollectionsGoeseleetal.Slidecredit:L.Lazebnik71.VisionformeasurementRealWhatisComputerVision?AutomaticunderstandingofimagesandvideoComputingpropertiesofthe3Dworldfromvisualdata(measurement)Algorithmsandrepresentationstoallowamachinetorecognizeobjects,people,scenes,andactivities.(perceptionandinterpretation)

8WhatisComputerVision?Automa2.Visionforperception,interpretationskywaterFerriswheelamusementparkCedarPoint12EtreetreetreecarouseldeckpeoplewaitinginlinerideriderideumbrellaspedestriansmaxairbenchtreeLakeEriepeoplesittingonrideObjectsActivitiesScenesLocationsText/writingFacesGesturesMotionsEmotions…TheWickedTwister92.Visionforperception,inteWhatisComputerVision?AutomaticunderstandingofimagesandvideoComputingpropertiesofthe3Dworldfromvisualdata(measurement)Algorithmsandrepresentationstoallowamachinetorecognizeobjects,people,scenes,andactivities.(perceptionandinterpretation)Algorithmstomine,search,andinteractwithvisualdata(searchandorganization)

10WhatisComputerVision?Automa3.Visionforsearchandorganization113.VisionforsearchandorganComponentsofacomputervisionsystemLightingSceneCameraComputerSceneInterpretationSrinivasaNarasimhan’sslide12ComponentsofacomputervisioComputervisionvshumanvisionWhatweseeWhatacomputersees13ComputervisionvshumanvisioVisionisreallyhardVisionisanamazingfeatofnaturalintelligence Visualcortexoccupiesabout50%ofbrainMorehumanbraindevotedtovisionthananythingelseIsthataqueenorabishop?14VisionisreallyhardVisionisVisionismultidisciplinaryFromwikiComputerGraphicsHCI15VisionismultidisciplinaryFrWhycomputervisionmattersSafetyHealthSecurityComfortAccessFun16WhycomputervisionmattersSafAlittlestoryaboutComputerVisionIn1966,MarvinMinskyatMITaskedhisundergraduatestudentGeraldJaySussmanto“spendthesummerlinkingacameratoacomputerandgettingthecomputertodescribewhatitsaw”.Wenowknowthattheproblemisslightlymoredifficultthanthat.(Szeliski2009,ComputerVision)17AlittlestoryaboutComputerRidiculouslybriefhistoryofcomputervision1966:Minskyassignscomputervisionasanundergraduatesummerproject1960’s:interpretationofsyntheticworlds1970’s:someprogressoninterpretingselectedimages1980’s:ANNscomeandgo;shifttowardgeometryandincreasedmathematicalrigor1990’s:facerecognition;statisticalanalysisinvogue2000’s:broaderrecognition;largeannotateddatasetsavailable;videoprocessingstarts2030’s:robotuprising?Guzman‘68OhtaKanade‘78TurkandPentland‘91Ridiculouslybriefhistoryof1919Whystudycomputervision?MillionsofimagesbeingcapturedallthetimeLotsofusefulapplicationsThenextslidesshowthecurrentstateoftheartSource:S.LazebnikWhystudycomputervision?MilFlickr1billion2billion3billion4billion5billion6billionFlickr1billion2billion3biOtherphotosharingsites10billion20billion50billion30billion40billionOtherphotosharingsites10b…andgrowingFlickr:>1.7millionphotos/dayFacebook:>100millionphotos/dayYouTube:>35hoursofvideoeveryminute~57billionphotoswillbetaken(US)in2010/windows_live/b/windowslive/archive/2010/04/09/what-to-do-with-57-billion-photos.aspx(asofNovember2010)(comparewith~17billionnegativesexposedin1996)(asofFebruary2010)…andgrowingFlickr:>1.7milHowvisionisusednowExamplesofstate-of-the-art24HowvisionisusednowExamples1.Opticalcharacterrecognition(OCR)Digitrecognition,AT&Tlabs/~yann/TechnologytoconvertscanneddocstotextIfyouhaveascanner,itprobablycamewithOCRsoftwareLicenseplatereaders/wiki/Automatic_number_plate_recognition251.Opticalcharacterrecogniti2.FacedetectionManynewdigitalcamerasnowdetectfacesCanon,Sony,Fuji,…262.FacedetectionManynewdigi3.SmiledetectionSonyCyber-shot®T70DigitalStillCamera273.SmiledetectionSonyCyber-s4.3DfromthousandsofimagesBuildingRomeinaDay:Agarwaletal.200928TheoldcityofDubrovnik,4,619images,3,485,717points4.3Dfromthousandsofimages5.Objectrecognition(insupermarkets)LaneHawkbyEvolutionRobotics“Asmartcameraisflush-mountedinthecheckoutlane,continuouslywatchingforitems.Whenanitemisdetectedandrecognized,thecashierverifiesthequantityofitemsthatwerefoundunderthebasket,andcontinuestoclosethetransaction.Theitemcanremainunderthebasket,andwithLaneHawk,youareassuredtogetpaidforit…“295.Objectrecognition(insupe6.Vision-basedbiometrics“HowtheAfghanGirlwasIdentifiedbyHerIrisPatterns”NationalGeographic306.Vision-basedbiometrics“How7.ForensicsSource:NayarandNishino,“EyesforRelighting”7.ForensicsSource:NayarandSource:NayarandNishino,“EyesforRelighting”Source:NayarandNishino,“EySource:NayarandNishino,“EyesforRelighting”Source:NayarandNishino,“Ey8.Loginwithoutapassword…Fingerprintscannersonmanynewlaptops,

otherdevicesFacerecognitionsystemsnowbeginningtoappearmorewidely

/348.Loginwithoutapassword…Fi9.Objectrecognition(inmobilephones)Point&Find,NokiaGoogleGoggles359.Objectrecognition(inmobi10.VisioninspaceVisionsystems(JPL)usedforseveraltasksPanoramastitching3DterrainmodelingObstacledetection,positiontrackingFormore,read“ComputerVisiononMars”byMatthiesetal.NASA'SMarsExplorationRoverSpiritcapturedthiswestwardviewfromatop

alowplateauwhereSpiritspenttheclosingmonthsof2007.3610.VisioninspaceVisionsyst11.IndustrialrobotsVision-guidedrobotspositionnutrunnersonwheels3711.IndustrialrobotsVision-gu12.Mobilerobots/NASA’sMarsSpiritRover/wiki/Spirit_roverSaxenaetal.2008STAIRatStanford3812.Mobilerobotshttp://www.roTHANKYOUSUCCESS2022/10/2839可编辑THANKYOUSUCCESS2022/10/2213.MedicalimagingImageguidedsurgeryGrimsonetal.,MIT3DimagingMRI,CT4013.MedicalimagingImageguide14.Digitalcosmetics 4114.Digitalcosmetics 4115.InpaintingBertalmioetal.SIGGRAPH004215.InpaintingBertalmioetal.16.DebluringFergusetal.SIGGRAPH064316.DebluringFergusetal.SIG17.SportsSportvisionfirstdownlineNiceexplanationon/video.html4417.SportsSportvisionfirstdo18.SmartcarsMobileyeVisionsystemscurrentlyinhigh-endBMW,GM,VolvomodelsBy2010:70%ofcarmanufacturers.4518.SmartcarsMobileye4519.GooglecarsOct9,2010.

"GoogleCarsDriveThemselves,inTraffic".

TheNewYorkTimes.JohnMarkoffJune24,2011."Nevadastatelawpavesthewayfordriverlesscars".

FinancialPost.ChristineDobbyAug9,2011,"HumanerrorblamedafterGoogle'sdriverlesscarsparksfive-vehiclecrash".

TheStar

(Toronto)4619.GooglecarsOct9,2010.

"G20.InteractiveGames:KinectObjectRecognition:/watch?feature=iv&v=fQ59dXOo63oMario:/watch?v=8CTJL5lUjHg3D:/watch?v=7QrnwoO1-8ARobot:/watch?v=w8BmgtMKFbY4720.InteractiveGames:KinectOTheMatrixmovies,ESCEntertainment,XYZRGB,NRC21.Specialeffects:shapecapture48TheMatrixmovies,ESCEntertaPiratesoftheCarribean,IndustrialLightandMagic22.Specialeffects:motioncapture49PiratesoftheCarribean,InduComputerVisionandNearbyFieldsComputerGraphics:ModelstoImagesComp.Photography:ImagestoImagesComputerVision:ImagestoModels50ComputerVisionandNearbyFieOverviewofComputerVisionAlgorithm51Sowhatdohumanscareabout?OverviewofComputerVisionAlVerification:isthatabus?slidebyFeiFei,Fergus&Torralba52Verification:isthatabus?slDetection:aretherecars?slidebyFeiFei,Fergus&Torralba53Detection:aretherecars?slidIdentification:isthatapictureofMao?slidebyFeiFei,Fergus&Torralba54Identification:isthatapictObjectcategorizationskybuildingflagwallbannerbuscarsbusfacestreetlampslidebyFeiFei,Fergus&Torralba55ObjectcategorizationskybuildiSceneandcontextcategorizationoutdoorcitytraffic…slidebyFeiFei,Fergus&Torralba56SceneandcontextcategorizatiRough3Dlayout,depthordering57Rough3Dlayout,depthorderinOverviewofComputerVisionAlgorithmImageformationFeaturesGrouping&fittingMulti-viewgeometryRecognition&learningMotion&tracking58OverviewofComputerVisionAl1.ImageformationHowdoeslightin3dworldprojecttoform2dimages?591.ImageformationHowdoeslig2.FeaturesandfiltersTransforminganddescribingimages;textures,colors,edges602.FeaturesandfiltersTransfo3.Grouping&fitting[figfromShietal]Clustering,segmentation,fitting;whatpartsbelongtogether?613.Grouping&fitting[figfrom4.MultipleviewsHartleyandZissermanMulti-viewgeometry,matching,invariantfeatures,stereovisionFei-FeiLi624.MultipleviewsHartleyandZ5.RecognitionandlearningRecognizingobjectsandcategories,learningtechniques635.RecognitionandlearningRec6.MotionandtrackingTrackingobjects,videoanalysis,lowlevelmotion,opticalflow646.MotionandtrackingTrackingChallenges1:viewpointvariationMichelangelo1475-156465Challenges1:viewpointvariaChallenges2:illuminationslidecredit:S.Ullman66Challenges2:illuminationslidChallenges3:occlusionMagritte,195767Challenges3:occlusionMagrittChallenges4:scaleslidebyFeiFei,Fergus&Torralba68Challenges4:scaleslidebyFeChallenges5:deformationXu,Beihong194369Challenges5:deformationXu,BChallenges6:backgroundclutterKlimt,191370Challenges6:backgroundcluttChallenges7:objectintra-classvariationslidebyFei-Fei,Fergus&Torralba71Challenges7:objectintra-claChallenges8:localambiguityslidebyFei-Fei,Fergus&Torralba72Challenges8:localambiguityChallenges9:theworldbehindtheimage73Challenges9:theworldbehinChallenges10:complexityThousandstomillionsofpixelsinanimage3,000-30,000humanrecognizableobjectcategories30+degreesoffreedomintheposeofarticulatedobjects(humans)BillionsofimagesindexedbyGoogleImageSearch18billion+printsproducedfromdigitalcameraimagesin2004295.5millioncameraphonessoldin200574Challenges10:complexityThousKeepMoving…Ok,clearlythevisionproblemisdeepandchallenging…timetogiveup?Activeresearchareawithexcitingprogress!………………………75KeepMoving…Ok,clearlythevTHANKYOUSUCCESS2022/10/2876可编辑THANKYOUSUCCESS2022/10/22ComputerVision

(机器视觉)ImagebyComputerVision(机器视觉)ImagebyToday’sTalkWhatisComputerVision?WhyStudyComputerVision?HowVisionisUsedNow?OverviewofComputerVisionAlgorithmChallengesofComputerVisionQuestions78Today’sTalkWhatisComputerVWhatiscomputervision?Terminator279Terminator5Whatiscomputervision?TerminEverypicturetellsastory80GoalofcomputervisionistowritecomputerprogramsthatcaninterpretimagesEverypicturetellsastory4GoCancomputersmatch(orbeat)humanvision?81Cancomputersmatch(orbeat)WhatisComputerVision?AutomaticunderstandingofimagesandvideoComputingpropertiesofthe3Dworldfromvisualdata(measurement)

82WhatisComputerVision?Automa

1.VisionformeasurementReal-timestereoStructurefrommotionNASAMarsRoverPollefeysetal.Multi-viewstereofor

communityphotocollectionsGoeseleetal.Slidecredit:L.Lazebnik831.VisionformeasurementRealWhatisComputerVision?AutomaticunderstandingofimagesandvideoComputingpropertiesofthe3Dworldfromvisualdata(measurement)Algorithmsandrepresentationstoallowamachinetorecognizeobjects,people,scenes,andactivities.(perceptionandinterpretation)

84WhatisComputerVision?Automa2.Visionforperception,interpretationskywaterFerriswheelamusementparkCedarPoint12EtreetreetreecarouseldeckpeoplewaitinginlinerideriderideumbrellaspedestriansmaxairbenchtreeLakeEriepeoplesittingonrideObjectsActivitiesScenesLocationsText/writingFacesGesturesMotionsEmotions…TheWickedTwister852.Visionforperception,inteWhatisComputerVision?AutomaticunderstandingofimagesandvideoComputingpropertiesofthe3Dworldfromvisualdata(measurement)Algorithmsandrepresentationstoallowamachinetorecognizeobjects,people,scenes,andactivities.(perceptionandinterpretation)Algorithmstomine,search,andinteractwithvisualdata(searchandorganization)

86WhatisComputerVision?Automa3.Visionforsearchandorganization873.VisionforsearchandorganComponentsofacomputervisionsystemLightingSceneCameraComputerSceneInterpretationSrinivasaNarasimhan’sslide88ComponentsofacomputervisioComputervisionvshumanvisionWhatweseeWhatacomputersees89ComputervisionvshumanvisioVisionisreallyhardVisionisanamazingfeatofnaturalintelligence Visualcortexoccupiesabout50%ofbrainMorehumanbraindevotedtovisionthananythingelseIsthataqueenorabishop?90VisionisreallyhardVisionisVisionismultidisciplinaryFromwikiComputerGraphicsHCI91VisionismultidisciplinaryFrWhycomputervisionmattersSafetyHealthSecurityComfortAccessFun92WhycomputervisionmattersSafAlittlestoryaboutComputerVisionIn1966,MarvinMinskyatMITaskedhisundergraduatestudentGeraldJaySussmanto“spendthesummerlinkingacameratoacomputerandgettingthecomputertodescribewhatitsaw”.Wenowknowthattheproblemisslightlymoredifficultthanthat.(Szeliski2009,ComputerVision)93AlittlestoryaboutComputerRidiculouslybriefhistoryofcomputervision1966:Minskyassignscomputervisionasanundergraduatesummerproject1960’s:interpretationofsyntheticworlds1970’s:someprogressoninterpretingselectedimages1980’s:ANNscomeandgo;shifttowardgeometryandincreasedmathematicalrigor1990’s:facerecognition;statisticalanalysisinvogue2000’s:broaderrecognition;largeannotateddatasetsavailable;videoprocessingstarts2030’s:robotuprising?Guzman‘68OhtaKanade‘78TurkandPentland‘91Ridiculouslybriefhistoryof9519Whystudycomputervision?MillionsofimagesbeingcapturedallthetimeLotsofusefulapplicationsThenextslidesshowthecurrentstateoftheartSource:S.LazebnikWhystudycomputervision?MilFlickr1billion2billion3billion4billion5billion6billionFlickr1billion2billion3biOtherphotosharingsites10billion20billion50billion30billion40billionOtherphotosharingsites10b…andgrowingFlickr:>1.7millionphotos/dayFacebook:>100millionphotos/dayYouTube:>35hoursofvideoeveryminute~57billionphotoswillbetaken(US)in2010/windows_live/b/windowslive/archive/2010/04/09/what-to-do-with-57-billion-photos.aspx(asofNovember2010)(comparewith~17billionnegativesexposedin1996)(asofFebruary2010)…andgrowingFlickr:>1.7milHowvisionisusednowExamplesofstate-of-the-art100HowvisionisusednowExamples1.Opticalcharacterrecognition(OCR)Digitrecognition,AT&Tlabs/~yann/TechnologytoconvertscanneddocstotextIfyouhaveascanner,itprobablycamewithOCRsoftwareLicenseplatereaders/wiki/Automatic_number_plate_recognition1011.Opticalcharacterrecogniti2.FacedetectionManynewdigitalcamerasnowdetectfacesCanon,Sony,Fuji,…1022.FacedetectionManynewdigi3.SmiledetectionSonyCyber-shot®T70DigitalStillCamera1033.SmiledetectionSonyCyber-s4.3DfromthousandsofimagesBuildingRomeinaDay:Agarwaletal.2009104TheoldcityofDubrovnik,4,619images,3,485,717points4.3Dfromthousandsofimages5.Objectrecognition(insupermarkets)LaneHawkbyEvolutionRobotics“Asmartcameraisflush-mountedinthecheckoutlane,continuouslywatchingforitems.Whenanitemisdetectedandrecognized,thecashierverifiesthequantityofitemsthatwerefoundunderthebasket,andcontinuestoclosethetransaction.Theitemcanremainunderthebasket,andwithLaneHawk,youareassuredtogetpaidforit…“1055.Objectrecognition(insupe6.Vision-basedbiometrics“HowtheAfghanGirlwasIdentifiedbyHerIrisPatterns”NationalGeographic1066.Vision-basedbiometrics“How7.ForensicsSource:NayarandNishino,“EyesforRelighting”7.ForensicsSource:NayarandSource:NayarandNishino,“EyesforRelighting”Source:NayarandNishino,“EySource:NayarandNishino,“EyesforRelighting”Source:NayarandNishino,“Ey8.Loginwithoutapassword…Fingerprintscannersonmanynewlaptops,

otherdevicesFacerecognitionsystemsnowbeginningtoappearmorewidely

/1108.Loginwithoutapassword…Fi9.Objectrecognition(inmobilephones)Point&Find,NokiaGoogleGoggles1119.Objectrecognition(inmobi10.VisioninspaceVisionsystems(JPL)usedforseveraltasksPanoramastitching3DterrainmodelingObstacledetection,positiontrackingFormore,read“ComputerVisiononMars”byMatthiesetal.NASA'SMarsExplorationRoverSpiritcapturedthiswestwardviewfromatop

alowplateauwhereSpiritspenttheclosingmonthsof2007.11210.VisioninspaceVisionsyst11.IndustrialrobotsVision-guidedrobotspositionnutrunnersonwheels11311.IndustrialrobotsVision-gu12.Mobilerobots/NASA’sMarsSpiritRover/wiki/Spirit_roverSaxenaetal.2008STAIRatStanford11412.Mobilerobotshttp://www.roTHANKYOUSUCCESS2022/10/28115可编辑THANKYOUSUCCESS2022/10/2213.MedicalimagingImageguidedsurgeryGrimsonetal.,MIT3DimagingMRI,CT11613.MedicalimagingImageguide14.Digitalcosmetics 11714.Digitalcosmetics 4115.InpaintingBertalmioetal.SIGGRAPH0011815.InpaintingBertalmioetal.16.DebluringFergusetal.SIGGRAPH0611916.DebluringFergusetal.SIG17.SportsSportvisionfirstdownlineNiceexplanationon/video.html12017.SportsSportvisionfirstdo18.SmartcarsMobileyeVisionsystemscurrentlyinhigh-endBMW,GM,VolvomodelsBy2010:70%ofcarmanufacturers.12118.SmartcarsMobileye4519.GooglecarsOct9,2010.

"GoogleCarsDriveThemselves,inTraffic".

TheNewYorkTimes.JohnMarkoffJune24,2011."Nevadastatelawpavesthewayfordriverlesscars".

FinancialPost.ChristineDobbyAug9,2011,"HumanerrorblamedafterGoogle'sdriverlesscarsparksfive-vehiclecrash".

TheStar

(Toronto)12219.GooglecarsOct9,2010.

"G20.InteractiveGames:KinectObjectRecognition:/watch?feature=iv&v=fQ59dXOo63oMario:/watch?v=8CTJL5lUjHg3D:/watch?v=7QrnwoO1-8ARobot:/watch?v=w8BmgtMKFbY12320.InteractiveGames:KinectOTheMatrixmovies,ESCEntertainment,XYZRGB,NRC21.Specialeffects:shapecapture124TheMatrixmovies,ESCEntertaPiratesoftheCarribean,IndustrialLightandMagic22.Specialeffects:motioncapture125PiratesoftheCarribean,InduComputerVisionandNearbyFieldsComputerGraphics:ModelstoImagesComp.Photography:ImagestoImagesComputerVision:ImagestoModels126ComputerVisionandNearbyFieOverviewofComputerVisionAlgorithm127Sowhatdohumanscareabout?OverviewofComputerVisionAlVerification:isthatabus?slidebyFeiFei,Fergus&Torralba128Verification:isthatabus?slDetection:aretherecars?slidebyFeiFei,Fergus&Torralba129Detection:aretherecars?slidIdentification:isthatapictureofMao?slidebyFeiFei,Fergus&Torralba130Identification:isthatapictObjectcategorizationskybuildingflagwallbannerbuscarsbusfacestreetlampslidebyFeiFei,Fergus&Torralba131ObjectcategorizationskybuildiSceneandcontextcategorizationoutdoorcitytraffic…slidebyFeiFei,Fergus&Torralba132SceneandcontextcategorizatiRough3Dlayout,depthordering133Rough3Dlayout,depthorderinOverviewofComput