外文文献及翻译基于视觉的矿井救援机器人场景识别Scenerecognitionforminerescuerobotlocalizationbasedonvision

1、附录 英文原文scene recognition for mine rescue robot localization based on visionabstract：a new scene recognition system was presented based on fuzzy logic and hidden markov model(hmm) that can be applied in mine rescue robot localization during emergencies. the system uses monocular camera to acquire omn

2、i-directional images of the mine environment where the robot locates. by adopting center-surround difference method, the salient local image regions are extracted from the images as natural landmarks. these landmarks are organized by using hmm to represent the scene where the robot is, and fuzzy log

3、ic strategy is used to match the scene and landmark. by this way, the localization problem, which is the scene recognition problem in the system, can be converted into the evaluation problem of hmm. the contributions of these skills make the system have the ability to deal with changes in scale, 2d

4、rotation and viewpoint. the results of experiments also prove that the system has higher ratio of recognition and localization in both static and dynamic mine environments.key words: robot location; scene recognition; salient image; matching strategy; fuzzy logic; hidden markov model1 introductionse

5、arch and rescue in disaster area in the domain of robot is a burgeoning and challenging subject1. mine rescue robot was developed to enter mines during emergencies to locate possible escape routes for those trapped inside and determine whether it is safe for human to enter or not. localization is a

6、fundamental problem in this field. localization methods based on camera can be mainly classified into geometric, topological or hybrid ones2. with its feasibility and effectiveness, scene recognition becomes one of the important technologies of topological localization.currently most scene recogniti

7、on methods are based on global image features and have two distinct stages: training offline and matching online.during the training stage, robot collects the images of the environment where it works and processes the images to extract global features that represent the scene. some approaches were u

8、sed to analyze the data-set of image directly and some primary features were found, such as the pca method 3. however, the pca method is not effective in distinguishing the classes of features. another type of approach uses appearance features including color, texture and edge density to represent t

9、he image. for example, zhou et al4 used multidimensional histograms to describe global appearance features. this method is simple but sensitive to scale and illumination changes. in fact, all kinds of global image features are suffered from the change of environment.lowe 5 presented a sift method th

10、at uses similarity invariant descriptors formed by characteristic scale and orientation at interest points to obtain the features. the features are invariant to image scaling, translation, rotation and partially invariant to illumination changes. but sift may generate 1 000 or more interest points,

11、which may slow down the processor dramatically.during the matching stage, nearest neighbor strategy(nn) is widely adopted for its facility and intelligibility6. but it cannot capture the contribution of individual feature for scene recognition. in experiments, the nn is not good enough to express th

12、e similarity between two patterns. furthermore, the selected features can not represent the scene thoroughly according to the state-of-art pattern recognition, which makes recognition not reliable7.so in this work a new recognition system is presented, which is more reliable and effective if it is u

13、sed in a complex mine environment. in this system, we improve the invariance by extracting salient local image regions as landmarks to replace the whole image to deal with large changes in scale, 2d rotation and viewpoint. and the number of interest points is reduced effectively, which makes the pro

14、cessing easier. fuzzy recognition strategy is designed to recognize the landmarks in place of nn, which can strengthen the contribution of individual feature for scene recognition. because of its partial information resuming ability, hidden markov model is adopted to organize those landmarks, which

15、can capture the structure or relationship among them. so scene recognition can be transformed to the evaluation problem of hmm, which makes recognition robust.2 salient local image regions detectionresearches on biological vision system indicate that organism (like drosophila) often pays attention t

16、o certain special regions in the scene for their behavioral relevance or local image cues while observing surroundings 8. these regions can be taken as natural landmarks to effectively represent and distinguish different environments. inspired by those, we use center-surround difference method to de

17、tect salient regions in multi-scale image spaces. the opponencies of color and texture are computed to create the saliency map.follow-up, sub-image centered at the salient position in s is taken as the landmark region. the size of the landmark region can be decided adaptively according to the change

18、s of gradient orientation of the local image 11.mobile robot navigation requires that natural landmarks should be detected stably when environments change to some extent. to validate the repeatability on landmark detection of our approach, we have done some experiments on the cases of scale, 2d rota

19、tion and viewpoint changes etc. fig.1 shows that the door is detected for its saliency when viewpoint changes. more detailed analysis and results about scale and rotation can be found in our previous works12.3 scene recognition and localizationdifferent from other scene recognition systems, our syst

20、em doesnt need training offline. in other words, our scenes are not classified in advance. when robot wanders, scenes captured at intervals of fixed time are used to build the vertex of a topological map, which represents the place where robot locates. although the maps geometric layout is ignored b

21、y the localization system, it is useful for visualization and debugging13 and beneficial to path planning. so localization means searching the best match of current scene on the map. in this paper hidden markov model is used to organize the extracted landmarks from current scene and create the verte

22、x of topological map for its partial information resuming ability.resembled by panoramic vision system, robot looks around to get omni-images. from fig.1 experiment on viewpoint changeseach image, salient local regions are detected and formed to be a sequence, named as landmark sequence whose order

23、is the same as the image sequence. then a hidden markov model is created based on the landmark sequence involving k salient local image regions, which is taken as the description of the place where the robot locates. in our system evi-d70 camera has a view field of 170. considering the overlap effec

24、t, we sample environment every 45 to get 8 images. let the 8 images as hidden state si (1i8), the created hmm can be illustrated by fig.2. the parameters of hmm, aij and bjk, are achieved by learning, using baulm-welch algorithm14. the threshold of convergence is set as 0.001.as for the edge of topo

25、logical map, we assign it with distance information between two vertices. the distances can be computed according to odometry readings.fig.2 hmm of environmentto locate itself on the topological map, robot must run its eye on environment and extract a landmark sequence l1 lk , then search the map fo

26、r the best matched vertex (scene). different from traditional probabilistic localization15, in our system localization problem can be converted to the evaluation problem of hmm. the vertex with the greatest evaluation value, which must also be greater than a threshold, is taken as the best matched v

27、ertex, which indicates the most possible place where the robot is.4 match strategy based on fuzzy logicone of the key issues in image match problem is to choose the most effective features or descriptors to represent the original image. due to robot movement, those extracted landmark regions will ch

28、ange at pixel level. so, the descriptors or features chosen should be invariant to some extent according to the changes of scale, rotation and viewpoint etc. in this paper, we use 4 features commonly adopted in the community that are briefly described as follows.go: gradient orientation. it has been

29、 proved that illumination and rotation changes are likely to have less influence on it5.asm and ent: angular second moment and entropy, which are two texture descriptors.h: hue, which is used to describe the fundamental information of the image.another key issue in match problem is to choose a good

30、match strategy or algorithm. usually nearest neighbor strategy (nn) is used to measure the similarity between two patterns. but we have found in the experiments that nn cant adequately exhibit the individual descriptor or features contribution to similarity measurement. as indicated in fig.4, the in

31、put image fig.4(a) comes from different view of fig.4(b). but the distance between figs.4(a) and (b) computed by jefferey divergence is larger than fig.4(c).to solve the problem, we design a new match algorithm based on fuzzy logic for exhibiting the subtle changes of each features. the algorithm is

32、 described as below.and the landmark in the database whose fused similarity degree is higher than any others is taken as the best match. the match results of figs.2(b) and (c) are demonstrated by fig.3. as indicated, this method can measure the similarity effectively between two patterns.fig.3 simil

33、arity computed using fuzzy strategy5 experiments and analysisthe localization system has been implemented on a mobile robot, which is built by our laboratory. the vision system is composed of a ccd camera and a frame-grabber ivc-4200. the resolution of image is set to be 400320 and the sample freque

34、ncy is set to be 10 frames/s. the computer system is composed of 1 ghz processor and 512 m memory, which is carried by the robot. presently the robot works in indoor environments.because hmm is adopted to represent and recognize the scene, our system has the ability to capture the discrimination abo

35、ut distribution of salient local image regions and distinguish similar scenes effectively. table 1 shows the recognition result of static environments including 5 laneways and a silo. 10 scenes are selected from each environment and hmms are created for each scene. then 20 scenes are collected when

36、the robot enters each environment subsequently to match the 60 hmms above.in the table, “truth” means that the scene to be localized matches with the right scene (the evaluation value of hmm is 30% greater than the second high evaluation). “uncertainty” means that the evaluation value of hmm is grea

37、ter than the second high evaluation under 10%. “error match” means that the scene to be localized matches with the wrong scene. in the table, the ratio of error match is 0. but it is possible that the scene to be localized cant match any scenes and new vertexes are created. furthermore, the “ratio o

38、f truth” about silo is lower because salient cues are fewer in this kind of environment.in the period of automatic exploring, similar scenes can be combined. the process can be summarized as: when localization succeeds, the current landmark sequence is added to the accompanying observation sequence

39、of the matched vertex un-repeatedly according to their orientation (including the angle of the image from which the salient local region and the heading of the robot come). the parameters of hmm are learned again.compared with the approaches using appearance features of the whole image (method 2, m2

40、), our system (m1) uses local salient regions to localize and map, which makes it have more tolerance of scale, viewpoint changes caused by robots movement and higher ratio of recognition and fewer amount of vertices on the topological map. so, our system has better performance in dynamic environmen

41、t. these can be seen in table 2. laneways 1, 2, 4, 5 are in operation where some miners are working, which puzzle the robot.6 conclusions1) salient local image features are extracted to replace the whole image to participate in recognition, which improve the tolerance of changes in scale, 2d rotatio

42、n and viewpoint of environment image.2) fuzzy logic is used to recognize the local image, and emphasize the individual features contribution to recognition, which improves the reliability of landmarks.3) hmm is used to capture the structure or relationship of those local images, which converts the s

43、cene recognition problem into the evaluation problem of hmm.4) the results from the above experiments demonstrate that the mine rescue robot scene recognition system has higher ratio of recognition and localization. future work will be focused on using hmm to deal with the uncertainty of localizatio

44、n.中文翻译基于视觉的矿井救援机器人场景识别摘要：基于模糊逻辑和隐马尔可夫模型（hmm），论文提出了一个新的场景识别系统，可应用于紧急情况下矿山救援机器人的定位。该系统使用单眼相机获取机器人所处位置的全方位的矿井环境图像。通过采用中心环绕差分法，从图像中提取突出的位置图像区域作为自然的位置标志。这些标志通过使用hmm有机组织起来代表机器人坐在场景，模糊逻辑算法用来匹配场景和位置标志。通过这种方式，定位问题，即系统的现场识别问题，可以转化为对hmm的评价问题。这些技术贡献使系统具有处理比率变化、二维旋转和视角变化的能力。实验结果还证明，该系统在静态和动态矿山环境中都具有较高的识别和定位的成功率。

45、关键字：机器人定位；场景识别；突出图像；匹配算法；模糊逻辑；隐马尔可夫模型1 介绍在机器人领域搜索和救援灾区是一个新兴而富有挑战性的课题。矿井救援机器人的开发是为了在紧急情况下进入矿井为被困人员查找可能的逃生路线，并确定该线路是否安全。定位识别是这个领域的基本问题。基于摄像头的定位可以主要分为几何法、拓扑法或混合法。凭借其可行性和有效性，场景识别成为拓扑定位的重要技术之一。目前，大多数场景识别方法是基于全局图像特征，有两个不同的阶段：离线培训和在线匹配。在训练阶段，机器人收集其所工作环境的图像，并处理这些图像提取出能表征该场景的全局特征。一些方法直接分析图像数据得到一些基本特征，比如pca方法

46、。但是，pca方法是不能区分特征的类别。另一种方法使用外观特征包括颜色、纹理和边缘密度来表示图像。例如，周等人用多维直方图来描述全局外观特征。此方法简单，但对比率和光照变化敏感。事实上，各种全局图像特征，所受来自环境变化的影响。lowe提出了sift方法，该方法利用关注点尺度和方向所形成的描述的相似性获得特征。这些特征对于图像缩放、平移、旋转和局部光照不变是稳定的。但sift可能产生1 000个或更多的兴趣点，这可能使处理器大大减慢。在匹配阶段，近邻算法（nn）因其简单和可行而被广泛采用。但是它并不能捕捉到个别特征对场景识别的贡献。在实验中，nn在表达两种部分之间的相似性时效果并不足够好。此外

47、，所选的特征并不能彻底地按照国家模式识别标准表示场景，这使得识别结果不可靠。因此，在这些分析中提出了一种新的识别系统，如果使用在复杂的矿井环境中它将更加可靠和有效。在这个系统中，我们通过提取突出的图像局部区域作为位置标志用以替代整个图像，改善了信息的稳定性，从而处理比率、二维旋转和视角的变化。兴趣点数量有效减少，这使得处理更加容易。模糊识别算法用以识别邻近位置的位置标志，它可以增强个别特征对场景识别的作用。由于它的部分信息恢复能力，采用隐马尔可夫模型组织这些位置标志，它可以捕捉到的结构或标志之间的关系。因此，场景识别可以转化为对hmm评价问题，这使得识别具有鲁棒性。2 局部图像区域不变形的检测

48、生物视觉系统的研究表明，生物体（像果蝇）在观察周围环境时，经常因为他们的行为习惯注意场景中确定的特殊区域或者局部图像信息。这些区域可以当作天然的位置标志有效地表示和区别不同环境。受这些启示，我们利用中心环绕差分法检测多尺度图像空间突出的区域。计算颜色和纹理的相似度用以绘制突出区域的地图。随后，以地图突出位置为中心的分图像，被定义为位置标志区域。位置标志区域的大小可以根据该区域图像梯度方向的变化自适应决定。移动机器人的导航要求当环境有一定程度变化时自然位置标志能被稳定地检测出来。为了验证我们方法对位置标志检测的的可重复性，我们已经在图像比例、二维旋转和视角等变化时，做了一些实验。图1表明当视角变化时因为它的突出效果大门能被检测出来。关于比率和旋转更详细的分析和结果可以在我们以前的论文中发现。图1 关于视角变化的实验3 场景识别和定位与其他场景识别系统不同的是，我们的不需要离线培训。换句话说，在前进中，我们不必对场景分类。当机器人徘徊时，在固定时