计算机视觉-多视图几何

MultipleViewGeometry

inComputerVisionMarcPollefeysComp290-089MultipleViewGeometryabcA(a,b)

A(a,b)

cf(a,b,c)=0abc(a,b,c)

(a,b,c)(reconstruction)(calibration)(transfer)CourseobjectivesTounderstandthegeometricrelationsbetweenmultipleviewsofscenes.Tounderstandthegeneralprinciplesofparameterestimation.Tobeabletocomputesceneandcamerapropertiesfromrealworldimagesusingstate-of-the-artalgorithms.Relationtoothervision/imagecoursesFocusesongeometricaspectsNoimageprocessingComp254:ImageProcessinganAnalysisMostlyorthogonaltothiscourse,complementaryComp256:ComputerVision(fall2003)Willbemuchbroader,basedonnewbook:“ComputerVision:amodernapproach”DavidForsythandJeanPonceMaterialTextbook:

MultipleViewGeometryinComputerVision byRichardHartleyandAndrewZisserman CambridgeUniversityPressAlternativebook:

TheGeometryfromMultipleImages byOlivierFaugerasandQuan-TuanLuong MITPressOn-linetutorial:

/~marc/tutorial.pdf

/~marc/tutorial/Learningapproach

readtherelevantchaptersofthebooksand/orreadingassignementsbeforethecourse.

Inthecoursethematerialwillthenbecoveredindetailandmotivatedwithrealworldexamplesandapplications.

Smallhands-onassignementswillbeprovidedtogivestudentsa"feel"ofthepracticalaspects.Studentswillalsoreadandpresentsomeseminalpaperstoprovideacomplementaryviewonsomeofthecoveredtopics.Finally,therewillalsobeaprojectwherestudentswillimplementanalgorithmorapproachusingconceptscoveredbythecourse.

Gradedistribution

Classparticipation:20%Hands-onassignments:10%Paperpresentation:10%Implementationassignment/project:40%Final:20%ApplicationsMatchMovingComputecameramotionfromvideo(toregisterrealanvirtualobjectmotion)Applications3DmodelingContentBackground:Projectivegeometry(2D,3D),Parameterestimation,Algorithmevaluation.SingleView:Cameramodel,Calibration,SingleViewGeometry.TwoViews:EpipolarGeometry,3Dreconstruction,ComputingF,Computingstructure,Planeandhomographies.ThreeViews:TrifocalTensor,ComputingT.MoreViews:N-Linearities,Multipleviewreconstruction,Bundleadjustment,auto-calibration,DynamicSfM,Cheirality,DualityMultipleViewGeometrycourseschedule

(tentative)Jan.7,9Intro&motivationProjective2DGeometryJan.14,16(nocourse)Projective2DGeometryJan.21,23Projective3DGeometryParameterEstimationJan.28,30ParameterEstimationAlgorithmEvaluationFeb.4,6CameraModelsCameraCalibrationFeb.11,13SingleViewGeometryEpipolarGeometryFeb.18,203DreconstructionFund.MatrixComp.Feb.25,27StructureComp.Planes&HomographiesMar.4,6TrifocalTensorThreeViewReconstructionMar.18,20MultipleViewGeometryMultipleViewReconstructionMar.25,27BundleadjustmentPapersApr.1,3Auto-CalibrationPapersApr.8,10DynamicSfM

PapersApr.15,17CheiralityPapersApr.22,24DualityProjectDemosFastForward!Quickoverviewofwhatiscoming…BackgroundLareproductioninterdite(ReproductionProhibited),1937,RenéMagritte.Points,lines&conicsTransformationsCross-ratioandinvariantsProjective2DGeometryProjective3DGeometryPoints,lines,planesandquadricsTransformationsП∞,ω∞

andΩ∞

EstimationHowtocomputeageometricrelationfromcorrespondences,e.g.2DtrafoLinear(normalized),non-linearandMaximumLikelihoodEstimationRobust(RANSAC)EvaluationanderroranalysisHowgoodaretheresultswegetBoundsonperformanceCovariancepropagation &Monte-CarloestimationresidualerrorSingle-ViewGeometryTheCyclops,c.1914,Odilon

RedonCameraModelsMostlypinholecameramodelbutalsoaffinecameras,pushbroomcamera,…CameraCalibrationComputePgiven(m,M)(normalized)linear,MLE,…RadialdistortionMoreSingle-ViewGeometryProjectivecamerasand planes,lines,conicsandquadrics.CameracenterandcamerarotationCameracalibrationandvanishingpoints,calibratingconicandtheIACSingleViewMetrologyAntonioCriminisiTwo-ViewGeometryTheBirthofVenus(detail),c.1485,Sandro

BotticelliEpipolarGeometryFundamentalmatrixEssentialmatrixTwo-ViewReconstructionEpipolarGeometryComputation(normalized)linear:minimal:MLE:RANSAC…andautomatedtwoviewmatchingRectificationWarpimagestosimplifyepipolargeometryStructureComputationPoints:Linear,optimal,directoptimalAlsolinesandvanishingpointsPlanesandHomographiesRelationbetweenplaneandHgivenPandP’RelationbetweenHandF,HfromF,FfromHTheinfinityhomography

H∞Three-ViewGeometryTheBirthofVenus(detail),c.1485,Sandro

BotticelliTrifocalTensorThreeViewReconstruction(normalized)linearminimal(6points)MLE(GoldStandard)Multiple-ViewGeometryTheBirthofVenus(detail),c.1485,Sandro

BotticelliMultipleViewGeometryQuadrifocaltensor81parameters,butonly29DOF!MultipleViewReconstructionAffinefactorizationProjectivefactorizationMultipleViewReconstructionSequentialreconstructionBundleAdjustmentMaximumLikelyhoodEstimationforcompletestructureandmotionU1U2U3WTWVP1P2P3M12xm3xn(ingeneralmuchlarger)BundleAdjustmentMaximumLikelyhoodEstimationforcompletestructureandmotionWTVU-WV-1WT11xm3xnBundle