视觉里程计原理(二)特征匹配与追踪(LK光流法)

1、Machine Perception and Interaction Group (MPIGMPIG) Feature matching陈伟杰MPIG Seminar 0046Machine Perception and Interaction Group (MPIG)Feature Extraction Feature matchingCompute E or F for R|tDrawing pathThe main steps of Visual OdometryimagesparametersMachine Perception and Interaction Group (MPIG)

2、secondFeature matching rich feature descriptorsoptical flowPoint matching usingbrute-forceFLANNHorn-SchunckLucas-KanadeMachine Perception and Interaction Group (MPIG)rich feature descriptorsComparing each feature in the first set to each feature in the second setK-dimension treeK-means treebrute-for

3、ceFLANN (Fast Library for Approximate Nearest Neighbors)Machine Perception and Interaction Group (MPIG)rich feature descriptorsexampleFLANN FlannBasedMatcher matcher; std:vector matches; matcher.match(descriptors1,descriptors2,matches); Mat img_matches; drawMatches( img1, keypoints1, img2, keypoints

4、2, matches, img_matches ); /- Draw matches imshow(Matches, img_matches ); /- Show detected matchescodeMachine Perception and Interaction Group (MPIG)rich feature descriptorsexampleFLANNMachine Perception and Interaction Group (MPIG)Eliminate errorscode double max_dist = 0; double min_dist = 100; for

5、( int i=0; idescriptors1.rows; i+ ) double dist = matchesi.distance; if( dist max_dist ) max_dist = dist; /- Draw only good matches (i.e. whose distance is less than 2*min_dist ) std:vector good_matches; for( int i = 0; i descriptors1.rows; i+ ) if( matchesi.distance 2*min_dist ) good_matches.push_b

6、ack( matchesi); /- Draw only good matches Mat img_matches; drawMatches( img1, keypoints1, img2, keypoints2, good_matches, img_matches, Scalar:all(- 1), Scalar:all(-1),vector(), DrawMatchesFlags:NOT_DRAW_SINGLE_POINTS ); /- Show detected matches imshow( Good Matches, img_matches ); Machine Perception

7、 and Interaction Group (MPIG)Eliminate errorsexampleMachine Perception and Interaction Group (MPIG)optical flowWhats optical flow?Machine Perception and Interaction Group (MPIG)optical flow Brightness constancy Temporal persistence or small movements Spatial coherencethree assumptionsLucas-Kanade (L

8、K or KLT) 1 1 B. D. Lucas, T. Kanade, An Iterative Image Registration Technique with an Application to Stereo Vision.Machine Perception and Interaction Group (MPIG) Brightness Constancy AssumptionAssumptionstranslational model:Machine Perception and Interaction Group (MPIG)TaylorAssumptions2nd assum

9、ptionMachine Perception and Interaction Group (MPIG)Assumptions3rd assumptionMachine Perception and Interaction Group (MPIG)KLT algorithmgray value u is I(x,y)uvThe goal is to find v on J, where I(u) and J(v) are similarThe way is to compute dMachine Perception and Interaction Group (MPIG)KLT algori

10、thmMachine Perception and Interaction Group (MPIG)Pyramid ImplementationStandard KLT algorithm deal with small pixel displacementSolution for this is a pyramidal implementationMachine Perception and Interaction Group (MPIG)Pyramid ImplementationInitial guessMachine Perception and Interaction Group (

11、MPIG)Pyramid Implementationmatching error function:Machine Perception and Interaction Group (MPIG)Standard KLT algorithmDefine new images A and BWe write as follow for convenienceMachine Perception and Interaction Group (MPIG)Standard KLT algorithmMachine Perception and Interaction Group (MPIG)Stand

12、ard KLT algorithmMachine Perception and Interaction Group (MPIG)Standard KLT algorithmMachine Perception and Interaction Group (MPIG)Code of LK based openCVcalcOpticalFlowPyrLK () vector err; vector status; Size winSize=Size(21,21);/设定金字塔层搜索窗口尺寸 TermCriteria termcrit=TermCriteria(TermCriteria:COUNT+

13、TermCriteria:EPS, 30, 0.01);/指定光流法搜索算法收敛迭代的类型 calcOpticalFlowPyrLK(img1, img2, points1, points2, status, err, winSize, 3, termcrit, 0, 0.001);Address: Perception and Interaction Group (MPIG)Eliminate errorsGetting rid of points for which the KLT tracking failed or those who have gone outside the frame int indexCorrection = 0;/初始化参数 for( int i=0; istatus.size(); i+) Point2f pt = points2.at(i- indexCorrection); if (status.at(i) = 0)|(pt.x0)|(pt.y0) if(pt.x0)|(pt.y0) status.at(i) = 0;/将对应的这组光流置零,等同于未发现该光流 points1.erase (points1.begin() + (i - indexCorrection);/删除该点 points2.e