张浩-机器人操控中的计算机视觉

机器人操控中的计算机视觉ComputerVisioninRobotManipulationDorabotInc.蓝胖子机器人CTO张浩TableofContentsWhat’sthedifference?Typicaltasks:ObjectposeestimationToyproblems:RobotcompetitionReal-worldproblems加入“知识星球行业与管理资源”库，每日免费获取报告1、每月分享1000+份最新行业报告（涵盖科技、金融、教育、互联网、房地产、生物制药、医疗健康等最新行业）2、每月分享500+企业咨询管理文件（涵盖国内外著名咨询公司相关管理方案，企业运营制度等）3、每月分享500+科技类论文或者大师课件、笔记。4、与高端人士交流企业管理思想与方法。加入微信群，每日获取免费3+份报告1、扫一扫二维码；或者添加微信（微信号：Teamkon）2、加入报告分享群，备注：姓名+单位+业务领域

微信扫码加入“知识星球行业与管理资源”，获取更多行业报告、管理文案、大师笔记微信扫码二维码，免费报告轻松领What’sthedifference?2DV.S.3D2DV.S.3DTaskFocusGeneralmethodv.s.TailoredmethodComputerVisiontriestosolveproblemsinitsmostgeneralform.DifferentrobotmanipulationtaskshavedifferentassumptionsthatsuitfordifferentcombinationsofCVmethods,mostprobablywithsomesmallinnovationshereandthere.Atailoredsolutionisusuallyrequired.Howdoesrobotinteractwiththephysicalworld?PerceptionPlanningSceneDecisionMotionPlanningforallDOFMotionExecutionforallDOFPlanningSceneTypicalVisionTasksinRobotManipulationEnvironmentreconstruction:buildplanningsceneSelf-localization:localizerobotinsuchsceneObjectposeestimation:supplyinformationinsceneVisualservoing:copewitherrorinvision&tasksrequirereal-timereactionRGBD-SLAMMappingandlocalizingdonothavetohappensimultaneously,buttheyhavetobe3D.ObjectPoseEstimationGoal:Full6DobjectposeApproach1:Align3DCADmodelto3Dpointcloud.ICP(iterativeclosestpoint)isfrequentlyused.Approach2:2Ddescriptors(SIFT,SURFetc.),3Ddescriptors(FPFH),orsimpler2Dor3Dfeatures(colorgradient,edge,normal)basedcorrespondences.MOPED,LINEMODroughlyfallsintothiscategory.ICPMOPEDLINEMODLINEMODLINEMODLINEMODNoneofthemethodsareuniversal!GoodToyProblems:RobotCompetitionsAmazonPickingChallenge:Fullyautonomous,focusonpick&placeDARPARoboticChallenge:RemotecontrolwithunstablenetworkconnectionRobocup@Home:Fullyautonomous,withvariouskindsoftasksincludingmanipulationRobocup@Work:Fullyautonomous,focusonmobilemanipulationofsimpleshapedworkpiecesRoboticGraspingandManipulationCompetition:Differenttrackswithdifferentautomationlevel,focusingonmanipulationcapabilityofrobotichandAmazonPickingChallengeAmazonPickingChallengeAmazonPickingChallengeClutteredenvironment:multipleobjectsinnarrowspacesOcclusion:LimitedcamerapositionMissingdata:reflective/transparent/meshedsurfacesSmallobjects:FewdatapointsDeformableobjects:modelalignmentdoesn’tworkSpeed:TimelimitforoveralltaskUncontrolledlightingProbabilisticMulti-ClassSegmentationProbabilisticMulti-ClassSegmentationJonschkowski,R.,Eppner,C.,Höfer,S.,Martín-Martín,R.,&Brock,O.(2016).ProbabilisticMulti-ClassSegmentationfortheAmazonPickingChallenge.InIEEE/RSJInternationalConferenceonIntelligentRobotsandSystems.ProbabilisticMulti-ClassSegmentationMulti-viewSelf-supervisedDeepLearningfor6DPoseEstimationMulti-viewSelf-supervisedDeepLearningfor6DPoseEstimationZeng,A.,Yu,K.-T.,Song,S.,Suo,D.,Walker,E.,Rodriguez,A.,&Xiao,J.(2016).Multi-viewSelf-supervisedDeepLearningfor6DPoseEstimationintheAmazonPickingChallenge.Retrievedfrom/abs/1609.09475Multi-viewSelf-supervisedDeepLearningfor6DPoseEstimationMulti-viewSelf-supervisedDeepLearningfor6DPoseEstimationDoraPickerDoraPickerZhang,H.,Long,P.,Zhou,D.,Qian,Z.,Wang,Z.,Wan,W.,…Pan,J.(2016).DoraPicker:AnAutonomousPickingSystemforGeneralObjects,10./10.1109/COASE.2016.7743473DoraPicker(LINEMOD+ICP)RoboticGraspingandManipulationCompetitionBookinpreparation:RoboticGraspingandManipulationCompetition,Springer,2017.DataSets/bigbird//rutgers_apc_rgbd_dataset/~andyz/apc2016Enoughtoyproblem,howaboutreal-worldproblem?DorabotMobileManipulator:WarehousedemoLookingforchallenges?Joinustoworkonreal-worldproblems!dream@关注蓝胖子公众账号加入“知识星球行业与管理资源”库，每日免费获取报告1、每月分享1000+份最新行业报告（涵盖科技、金融、教育、互联网、房地产、生物制药、医疗健康等最新行业）2、每月分享500+企业咨询管理文件（涵盖国内外著名咨询公司相关管理方案，企业