(优选)机器视觉综述.课件

1、机器视觉综述第1页，共73页。Todays TalkWhat is Computer Vision?Why Study Computer Vision?How Vision is Used Now?Overview of Computer Vision AlgorithmChallenges of Computer VisionQuestions2第2页，共73页。What is computer vision?Terminator 23Terminator 5第3页，共73页。Every picture tells a story4Goal of computer vision is to

2、write computer programs that can interpret images第4页，共73页。Can computers match (or beat) human vision?5第5页，共73页。What is Computer Vision?Automatic understanding of images and videoComputing properties of the 3D world from visual data (measurement) 6第6页，共73页。 1. Vision for measurementReal-time stereoSt

3、ructure from motionNASA Mars RoverPollefeys et al.Multi-view stereo forcommunity photo collectionsGoesele et al.Slide credit: L. Lazebnik7第7页，共73页。What is Computer Vision?Automatic understanding of images and videoComputing properties of the 3D world from visual data (measurement)Algorithms and repr

4、esentations to allow a machine to recognize objects, people, scenes, and activities. (perception and interpretation) 8第8页，共73页。2. Vision for perception, interpretationskywaterFerris wheelamusement parkCedar Point12 Etreetreetreecarouseldeckpeople waiting in linerideriderideumbrellaspedestriansmaxair

5、benchtreeLake Eriepeople sitting on rideObjectsActivitiesScenesLocationsText / writingFacesGesturesMotionsEmotionsThe Wicked Twister9第9页，共73页。What is Computer Vision?Automatic understanding of images and videoComputing properties of the 3D world from visual data (measurement)Algorithms and represent

6、ations to allow a machine to recognize objects, people, scenes, and activities. (perception and interpretation)Algorithms to mine, search, and interact with visual data (search and organization) 10第10页，共73页。3. Vision for search and organization11第11页，共73页。Components of a computer vision systemLighti

7、ngSceneCameraComputer Scene InterpretationSrinivasa Narasimhans slide12第12页，共73页。Computer vision vs human visionWhat we seeWhat a computer sees13第13页，共73页。Vision is really hardVision is an amazing feat of natural intelligenceVisual cortex occupies about 50% of brainMore human brain devoted to vision

8、 than anything elseIs that a queen or a bishop?14第14页，共73页。Vision is multidisciplinary From wikiComputer GraphicsHCI15第15页，共73页。Why computer vision mattersSafetyHealthSecurityComfortAccessFun16第16页，共73页。A little story about Computer VisionIn 1966, Marvin Minsky at MIT asked his undergraduate student

9、 Gerald Jay Sussman to “spend the summer linking a camera to acomputer and getting the computer to describe what it saw”. We now know that the problem is slightly more difficult than that. (Szeliski 2009, Computer Vision)17第17页，共73页。Ridiculously brief history of computer vision1966: Minsky assigns c

10、omputer vision as an undergraduate summer project1960s: interpretation of synthetic worlds1970s: some progress on interpreting selected images1980s: ANNs come and go; shift toward geometry and increased mathematical rigor1990s: face recognition; statistical analysis in vogue2000s: broader recognitio

11、n; large annotated datasets available; video processing starts2030s: robot uprising?Guzman 68Ohta Kanade 78Turk and Pentland 91第18页，共73页。19第19页，共73页。 Why study computer vision?Millions of images being captured all the timeLots of useful applicationsThe next slides show the current state of the artSo

12、urce: S. Lazebnik第20页，共73页。 Flickr1 billion2 billion3 billion4 billion5 billion6 billion第21页，共73页。 Other photo sharing sites10 billion20 billion50 billion30 billion40 billion第22页，共73页。 and growingFlickr: 1.7 million photos / dayFacebook: 100 million photos / dayYouTube: 35 hours of video every minut

13、e 57 billion photos will be taken (US) in 2010/windows_live/b/windowslive/archive/2010/04/09/what-to-do-with-57-billion-photos.aspx(as of November 2010)(compare with 17 billion negatives exposed in 1996)(as of February 2010)第23页，共73页。How vision is used nowExamples of state-of-the-art24第24页，共73页。1. O

14、ptical character recognition (OCR)Digit recognition, AT&T labs/yann/Technology to convert scanned docs to textIf you have a scanner, it probably came with OCR softwareLicense plate readers/wiki/Automatic_number_plate_recognition25第25页，共73页。2. Face detectionMany new digital cameras now detect facesCa

15、non, Sony, Fuji, 26第26页，共73页。3. Smile detectionSony Cyber-shot T70 Digital Still Camera 27第27页，共73页。4. 3D from thousands of imagesBuilding Rome in a Day: Agarwal et al. 200928The old city of Dubrovnik, 4,619 images, 3,485,717 points第28页，共73页。5. Object recognition (in supermarkets)LaneHawk by Evoluti

16、onRobotics“A smart camera is flush-mounted in the checkout lane, continuously watching for items. When an item is detected and recognized, the cashier verifies the quantity of items that were found under the basket, and continues to close the transaction. The item can remain under the basket, and wi

17、th LaneHawk, you are assured to get paid for it “29第29页，共73页。6. Vision-based biometrics“How the Afghan Girl was Identified by Her Iris Patterns” National Geographic30第30页，共73页。7. ForensicsSource: Nayar and Nishino, “Eyes for Relighting”第31页，共73页。Source: Nayar and Nishino, “Eyes for Relighting”第32页，共

18、73页。Source: Nayar and Nishino, “Eyes for Relighting”第33页，共73页。8. Login without a passwordFingerprint scanners on many new laptops, other devicesFace recognition systems now beginning to appear more widely/34第34页，共73页。9. Object recognition (in mobile phones)Point & Find, NokiaGoogle Goggles35第35页，共73

19、页。10. Vision in spaceVision systems (JPL) used for several tasksPanorama stitching3D terrain modelingObstacle detection, position trackingFor more, read “Computer Vision on Mars” by Matthies et al.NASAS Mars Exploration Rover Spirit captured this westward view from atop a low plateau where Spirit sp

20、ent the closing months of 2007. 36第36页，共73页。11. Industrial robotsVision-guided robots position nut runners on wheels37第37页，共73页。12. Mobile robots/NASAs Mars Spirit Rover/wiki/Spirit_roverSaxena et al. 2008STAIR at Stanford38第38页，共73页。13. Medical imagingImage guided surgeryGrimson et al., MIT3D imagi

21、ngMRI, CT39第39页，共73页。14. Digital cosmetics40第40页，共73页。15. InpaintingBertalmio et al. SIGGRAPH 0041第41页，共73页。16. DebluringFergus et al. SIGGRAPH 0642第42页，共73页。17. SportsSportvision first down lineNice explanation on /video.html43第43页，共73页。18. Smart carsMobileyeVision systems currently in high-end BMW

22、, GM, Volvo models By 2010: 70% of car manufacturers.44第44页，共73页。19. Google carsOct 9, 2010.Google Cars Drive Themselves, in Traffic.The New York Times. John MarkoffJune 24, 2011. Nevada state law paves the way for driverless cars.Financial Post. Christine DobbyAug 9, 2011, Human error blamed after

23、Googles driverless car sparks five-vehicle crash.The Star(Toronto)45第45页，共73页。20. Interactive Games: KinectObject Recognition: /watch?feature=iv&v=fQ59dXOo63oMario: /watch?v=8CTJL5lUjHg3D: /watch?v=7QrnwoO1-8ARobot: /watch?v=w8BmgtMKFbY46第46页，共73页。The Matrix movies, ESC Entertainment, XYZRGB, NRC21.

24、 Special effects: shape capture47第47页，共73页。Pirates of the Carribean, Industrial Light and Magic22. Special effects: motion capture48第48页，共73页。Computer Vision and Nearby FieldsComputer Graphics: Models to ImagesComp. Photography: Images to ImagesComputer Vision: Images to Models49第49页，共73页。Overview o

25、f Computer Vision Algorithm50So what do humans care about?第50页，共73页。Verification: is that a bus?slide by Fei Fei, Fergus & Torralba 51第51页，共73页。Detection: are there cars?slide by Fei Fei, Fergus & Torralba 52第52页，共73页。Identification: is that a picture of Mao?slide by Fei Fei, Fergus & Torralba 53第53

26、页，共73页。Object categorizationskybuildingflagwallbannerbuscarsbusfacestreet lampslide by Fei Fei, Fergus & Torralba 54第54页，共73页。Scene and context categorization outdoor city traffic slide by Fei Fei, Fergus & Torralba 55第55页，共73页。Rough 3D layout, depth ordering56第56页，共73页。Overview of Computer Vision A

27、lgorithmImage formationFeatures Grouping & fittingMulti-view geometryRecognition & learningMotion & tracking57第57页，共73页。1. Image formationHow does light in 3d world project to form 2d images?58第58页，共73页。2. Features and filtersTransforming and describing images; textures, colors, edges59第59页，共73页。3.

28、Grouping & fittingfig from Shi et alClustering, segmentation, fitting; what parts belong together?60第60页，共73页。4. Multiple viewsHartley and ZissermanMulti-view geometry, matching, invariant features, stereo visionFei-Fei Li61第61页，共73页。5. Recognition and learningRecognizing objects and categories, learning techniques62第62页，共73页。6. Motion and trackingTracking objects, video analysis, low level motion,