一种基于贪心搜索的实时多目标遮挡处理算法

1、ø 36 ø 3 2010 ¢ 3 $ k acta automatica sinicavol. 36, no. 3march, 2010%¼ 1 2 3 1¼ %$f¼¿f%fi, :%,¾q %Ç%¿. %ka$: 1) f%k½%; 2) Ǿ%¿$%, ½f%½ 3) ½ $, fiaf% (qr%$ 2), %½%¼%fi$, $%½. $i¿% ibm fi$fi%½

2、;f%$½. fi, , Ç, doi10.3724/sp.j.1004.2010.00375a greedy searching algorithm for multiple objecttracking and occlusion handlingyang tao1li jing2pan quan3zhang yan-ning1abstract this paper presents a novel real-time multiple object tracking algorithm, which contains three parts: regioncorrel

3、ation based foreground segmentation, merging-splitting based data association and greedy searching based occluded object localization. the main characteristics of the proposed algorithm are summarized as follows: 1) the multiple object tracking and occlusion handling problem is successfully changed

4、into an image classification problem with prior knowledge of object number and feature; 2) a highly efficient greedy searching method is presented to meet real-time capability; 3) it has good performance in expansibility, and it has no constraints about the number of occluded objects, the occlusion

5、ratio and the objectt s motion model. experiment results with hand labeled ibm database demonstrate that the method is effective and efficient.key words multiple object detection and tracking, occlusion handling, greedy searching, intelligent video surveillance%kª%fi¼fifl$%½fi, y,$a,&

6、#189;þÇ$%f. f%fi116 %$% aa$. r$, $þ%, $afi $%.%pf 5 ¼: 1) fi½ 2) fi¾q; 3) ¾qfi; 4) fi; 5) fi. $, ¼ 1) 5) %¼, ¼ 2) qr %¾, f®¼a%½¾ (¼ 3) þ®¼%¿ (¼ 4) %a.k¾, ¼¿f%lfffflfl. $q:, ¾q

7、 %¾ $Ç% , %a¼%½¿¾.1) ¾q %ø¼½%½¾, qr%, %½, qfl%¾qf%. l, q r%q$%, %qr%, ®¼ $. mckenna 13 bremond14 k½, ®,¾. piater 15 ¾k½¾$ 2008-10-22 fi$ 2009-01-21manuscript received october 22, 2008; accep

8、ted january 21,2009¾$q (863 ) (2009aa01z315), ¾ (60903126, 60872145, 60634030), ½i¼$ø (708085) supported by national high technology research and de-velopment program of china (863 program) (2009aa01z315), national natural science foundation of china (60903126,60872145, 6063

9、4030), and cultivation fund of the key scien- tific and technical innovation project, ministry of education of china (708085)1. ffli!$fa fp710129 2. fpÇ$i¼ fp 710071 3. ffli!$k fp 7101291. shaanxi key laboratory of speech and image informationprocessing, school of computer science, northwe

10、stern polytech- nical university, xir an 710129 2. school of telecommunica- tions engineering, xidian university, xir an 710071 3. school of automation, northwestern polytechnical university, xir an710129, q kalman ø%, ¾.¾q %¾fl%, ¼$¾q¾ , fl½%¼%,%. ¾

11、q%, ¾q%k½%. %a%®qr%$ 2, %:¢fi, %½¼%$, fl%k½%½$ %f%, f.2) Ç%¼$%k. ½qr %$, fl¾fi ($ 2) $%. %a%.%,mean shift . $% (®a,ka,½) Ç$ (®fi Ç,%l), %¢½ ¼%. , %½, ½$¿%¾½,%½%¼$, %

12、½f¾. %, $l¾, %fikfi, qrfl%, %$, f%, $%, . flk, ø ½%, %½½f%.a%$%¿,Ç$%½, %$k. ¾l¾,qr%, $%¾fia, f$fiqr%, %¾fi, %fl %½%. fl, $%$fifik. elgammal 16 %fi$% %, %$% % (fi,$,¶), %$ fi$%$, %, q¾. cucchiara10 :$fi$% $. beleznai

13、 11 mean shift fi¾%$¾Ç, q¾% mean shift . %, lffl: cucchiara 10 ½fi¾k, fk½%. flk,¾fi%¿½%, fi½%, %½f%. beleznai 11 ½$%ø lÇ, f, f%Ǿ mean shift %$.$lfl¾, %$%¼¿fi. ¾ q , ½fi%¾, %$aÇ%,

14、f%k½%, q¼%, ǽ$%¾¾%¿. ½qr%,%¼% þ%fi$, ½f%$½.$%p®f: ø 1 ;$% ÷ ø 2 ;$%¾q ; ø 3 ;Ç%; ø 4 ;$; ø 5 ;%.1 k 3 : 1) ø%;2) ¾q; 3) Ç%¿. ¼® 1 . ½pfifi,ø%$¾, fifi

15、:, :%k%k %, ¾%¾q. $qr%, ¾, q½ $qr%, ½ fi%¾q, Ç% , ¿$%. 1 Ç%fifig. 1 greedy searching based multiple object tracking and occlusion handling algorithm3 : Ç%fi3772 ¾q2.1 %ø (background subtraction)1722%f%, $ %pø%½$%, ø½,%,&

16、#174;fi, flfi$%f. ø%®¼%$%ø%. p½%ø%$,½, ,¾%17 ,%,a f tt %a d %$%½a c f :v+n u+n1.d(u, v) =f tt (x, y)(2n + 1)2y=vn x=un.1,q d(u, v) tc0,¼f =(3)$, tc = 0.1. $fi% ½%k, ½½a½%½%k%ka½%, %¼$. ÷ i b $p$ø, ½a

17、(u, v) % (2m +1)×(2m +1)¢kfi%fi®f:n c c (u, v) =$18 .f¾½%$, ,½¾,%,k½h%$f.$ ¾ % (gaussian mixture model, gmm)17 ¾ø%, q$% $rø. gmm ®f:¿$fiafi% x $, %½®f k ½$:m.m.(b(u + u, v + v)i (u + u, v + v)v=m u=m.m.m.b(u + u, v

18、+ v)2 ×,v=m u=m.m.m.k.i (u + u, v + v)2(4)p (xt ) = . i,t xt , µi,t , .,(1)v=m u=mi=1i,t%, fi (4) $%¾%. $, $k% øf%, %$.2.2¾q ti , i = 1, · · · , n1 $%¶,fi$ i,t ø i $ t %, %$.k.i=1 i,t = 1, (xt , µi,t , i,t ) % x t %øi $.1(xt , µi,t , .

19、) =i,t×n.i,t1(2) 2 | 2 d , j = 1, · · · , n $j2%. ¶ t d %, $ 23 $%$%¾¾q. fl%fl: m c .1) m t d $%fi½%, $%¶®, ¢k,½%½. %fi%, %k¾q$%$, fl, k:fl®%, r m .t 1 2 (xt µi,t ) . (xt µi,t )i,t1e(2)fi$ i = 1, · · ·

20、; , k , n $ xt %, f%¼, p r, g, b a, q$¼%½, $ .i,t = i i . $ gmm $2$%$ø, ®$fifi%¾$%ø$, %k. flk, f%¼% xt , %½, %fi%k, ¾½fi%¾%.fi¾½½fik¾ fl, fi$%½, %¾%$: ø%, , f¾ $%, :%fifi%qr%k, :$, l. fl½l, $ gmm

21、 %l, , a%$. ½fia (u, v), (2n + 1) × (2n + 1) ¢sti djm (i, j) = c(5)sti + sdj$, sti dj$¶ ti dj %:, sti sdj $ ti dj %k, fi c %kp, %$ c = 2, m (i, j) 0, 1. ¶®fl, %¾$ f%$fl½, fl$%¿½.2) c m , %$, t d % c . % c %$yfi 0, c ¼®f:a) ½ m , ½

22、 ti , $¼%fi jmax , q c %½ 1:fi¾q%, ½%½%¾ z , g% p (zg |h ) $k. 2 $f%¿, $ h fi$ w ti i=1,··· ,w%¾q, zg fi$ ti i=1,··· ,w ¾q$%$¾, %k%¾q%$. h r a%, ¾ zg % rw , %jmax = arg max m (i, j),j = 1, · · ·

23、, n1j, fi½% z flgc (i, jmax ) = c (i, jmax ) + 1(6)%½%.b) ½fi m , ½%dj , $¼% jmax , q c %½ 1:jmax = arg max m (i, j),i = 1, · · · , n2ic (imax , j) = c (imax , j) + 1(7)c) c , ® 2 %yfi, yfi½%, q m$%fi$yfi 0, a) fl%; ® c $yfi 2, $n.y c %, &#

24、182; t d % 3 : 1) %¶2) $%¶ 3) $%fi%. ½$%¶, % ,%,ø% þ¾q; ½$fi%, % $,%þø. fl, ®$%¶$%qr, fi¾qf.¾q %6, 23 ¾q %fi, ½fl %k,½,¼%,%, ¾ fla%¾. %, ¾ q, ¾¿f ¾¼$%¿, qr%$ 2 %, ¢%

25、, .3 %yfþ¾q %, $;%$Ç%, ¼$%k½%, qÇ% . f%, %¼, flkk%. l;%¾q, fi¾q¢. fl%, k%fi%. 2 $%fi¿fig. 2 multiple objects location in a merged foreground¾%½¾ z g , ¿$%: % vi i=1,··· ,w , %¼%. vi $, $%¼%. ti $%vi =

26、(8) t i¿fi¢%¾g , %$fi%fi½¾fi:z z 1 = arg max vi ,i 1, · · · , wi2 = arg max v ,i 1, · · · , w, v / v z ii1i(9).z = arg max vi ,i 1, · · · , w,kivi / v j j=1,··· ,k1®% vi 1, % p (zi |h )fi$, %¼%, vi , fl% p

27、 (zi |h ) %¼. l½, ti¼$qr$%fl%k%, ti % p (zi |h ) % vi $fl, fl, $¿%, q%$, f%½$½%¾fi¿:p (z 1 |h ) = max p (zi |h )p (z |h, z ) = max p (z |h, z )21i13 : Ç%fi379fi$, p (i (x)|ti ) fi i (x) ti %$%.fi (16) %, ½% %, ¼%$½% fi. ½½¿%&

28、#189;½½. %fi (16) %$y.p (z |h, z , · · · , z ) =k1k1max p (z |h, z , · · · , z )(10)i1k1fi$ i 1, · · · , w, zi / z j=1,··· ,k1 . j$a, ti % tj , fi (10) fi$, q%fi:¿%¾ z 1 , · · · , zk1 %, fl%, $fi¼%,ff

29、iq$k¼%¾.f¾, h $%fifl¾:p (z |h ) · · · p (z |h, z , · · · , z ) = p (z |h )1w1w1g(11)fi (10) fi (11) $, % p (zg |h )$k%. %h $%¾ z 1 , · · · , zk1 f%fi¾ xg+$k% z , Ç%, ¾$%fi¾ x g , qfi (16) $:g p (i (x)| ti )p (t

30、i ) % z ½%$%:1,x x wg . p (i (x)|t )p (t )1 = arg max p (zi |h ),i = 1, · · · , w(12)z s sis=kp (ti |i (x) =fi z , $k% p (i (x)| ti )p (ti ) (x), x1fi z :x +wgk .p (i (x)|ts )p (ts )s=kk = arg max p (zi |h, z1 , z2 , · · · , zk1 )(13)z i(17)$, i = 1, · ·

31、· , w, zi / z j=1,··· ,k1 . f½ (x) 0, 1, %f%fi, %¼fifi%. 3 $Ǽ. $q r%, ½fi%¾q h , ¾ q%, ¢ qr%,½. ¿%, $%¿%, fi (14),(15) (17) ti %, q%¿$, ¼%fifij% p (zi |h, z , · · · , z ) , ti1k1%k zi h l$%, q%. $%:p

32、 (zi |h, z , · · · , z ) = max(p (zi |fxc ), xc h1k1(14)$a xc %k zi ffxc%. % p (zi |fxc ) fi$f ti fxc %. ½, f½%½, $ fxc $fi% p (zi |fxc ):+¾ xg $, ½¾q¾$%fi¾¾. l¼, fi$%¿.4q$, vc+ flqf%fi$fi. $fi intel core2 cpu 1.66 ghz ø,

33、62; 2 g. ibm ¢fi$fi24 %fi, q½ 25 $%½¾½.4.1%½¼, ½f 3 ¾:1) %fi¾½ ;1.xk fxcp (zi |fxc ) = zwp (ti |i (x)(15)· zhfi$, i (x) fi$ xk %fifi%, zw zh$ zi %. fi i (x) ti % p (ti |i (x) $, p (ti |i (x) %¾%. qr%$½%¾, f¾k%½ p (t

34、i |i (x), ½fi, h ¢$¾$¾¿%¾ p (ti |i (x): p (i (x)| ti )p (ti ) p (ti |i (x) =(16)w. p (i (x)|ts )p (ts )s=k 3 Ǽfig. 3 flowchart of greedy searching algorithmfp2) k%½3) %½.fi¾$i¿, , fl$k% fi, ¢%, q½ pets2006 $ 25 % 9 %½¾

35、, %®f:false positive rate (fpr) =(18)fp + tn$, tp, tn, fp, fn $ true positive, truenegative, false positive, false negative $%, tg (total ground truth) %$%, tf (total frame) %. k%, $,%½. f¿ %, $¿$%: (occlusion error rate, oer), qr%fl:tptgtracker detection rate (trdr) =fpfalse ala

36、 rate (far) =fptp% =(19)detection rate (dr) =qr%tp + fntn4.2%½¾, kp%$fi, % h½r$%. fl, ¾½¼$fi½%fi¾. ½ % fl fi pets,ibm % ¢ fi24 , % caviar fi26 , ¾ % etiseo27 . ibm fi$fispecificity =fp + tntp + tnaccuracy =tftppositive prediction (pp) =tp + fptn

37、negative prediction (np) =fn + tnfnfalse negative rate (fnr) =fn + tp3 : Ç%fi381$, fl, $fi½Ç%¾.ibm %¼, fl, ½ 10 ibm ¢% ,¾$i¿, r% (ground truth). ¿%, ½, k $, qǼ%, ¿%fi$. $i¿f 10¾ 7 443 , $ 3 085 , qr 64 . 4 $ 10 $i¿

38、%.$, l. 6 (c) 6 (d) $, fi¾½% gmm %$ $%: ( 6 (c) 6 (d) $), flk, kfi%h$ ( 6 (c) 6 (d) $½$). 6 (e) 6 (f ) $%, ¾½a¼%, fi¾½½¾½, $%¼%$. 4 ibm ¢fi$ 10 $i½%fig. 4 hand labeled ground truth trajectory of ibmindoor surveillance video databa

39、se%, fi$¼% , ¾fl, q fl%¼, % tp,tn,fp,fn, %fi (18) (19) $. 5 øf% (ø 1 : p; ø 2 : gmm ; ø 3 : $)fig. 5 moving foreground segmentation results under similar background (the first row shows the input video, the second row displays the gmm results, and the third row dis

40、plays our results)4.3ø%$ ibm ¢fi, øfl½ø 2.1 ;¾, q ¾ø%fl. fi%fi%, 9 × 9 %d, 0.9. ½ ibm fi%$, ½p 320 × 240 %fifi, % 15 /.ø%k. 5 $øf%. k½, gmm f$ fia ( 5 ø 2 ), $¼ %$, fl%¿½. $½a%, y,k$, $%f½%, % (

41、5 ø 3 ). 6 $¢¼%$ ø%. $%, k,¼f$. %, :¢%fifi%qrfl%k, ¶®%fi¾½:(b) ø(b) background image(a) p(a) input image(d) gmm (d) gmm foreground image(c) gmm (c) gmm segmentation result(f ) $(f ) our foreground image(e) $(e) our segmentation result 6fig. 6,&#

42、248;f%moving foreground segmentation results under cast shadow and similar background4.4$, ®,mean shift %qr.$Ç%, %$¿$% 1 , fi (17)$¾q¢fi%, q 2 %$%½¾¿. ½%¼%$, fl¼$fi$fl%¾ ( 7 (c) $ø 3 fi), f%ya%$%, ½% ( 7 (c) $ø 4 fi). 7 $ 3 f

43、l$%. fl%, ¾q %6, 23 ¾. $¼, Ç, $¼$k (7 (a) (c) $ø 2 fiø 3 fi). 7 (c) $, 2%fl ( 7 (c) $ø 3 fi), kqr$%, fl% 7 fl$experiment results of two people tracking through occlusionfig. 7 8 $experiment results of three-people tracking through occlusionfig. 83 : Ç%fi3

44、83 8 $ 3 $f%. $ø 1 $p, ø 2 $ %. %fi, $ %qra, $%$fi$¼% 3 %¾ ( 8 $ #180, #222). 3 %¾q%, $%, ¾q %qr. $ Ç, ¼$k¿$ 3 % ( 8 $ #180, #222), q 3 $ ( 8 $ #204, #240). 25 fi (18) $%½ ¾½. 9 $½ ibm fi$% 10 , ¾ 7 443 %½ . ½fi, $f%: (f

45、alse negative rate, fnr) 0.027 (9), $%øf½%. %¼, %¾fi, f% ; ¼%, n $ m (m < n ) %, fø%$ . fi% (false positive rate, fpr) 0.077, , ibm fi$ fl¾, 10 fifi$¾qr64 , $f 50 , 0.22. flk, Ç, $%½, ½fi% fi 10 /.%, ¼%, Ç$½¾%¿. $, $i

46、¿% ibm ¢fi$fi½½¾ , ½f%$½. ¾%$,$%¾f, fi½¾% %, %p, % f%½.references1 senior a, hampapur a, tian y l, brown l, pankanti s, bolle r. appearance models for occlusion handling. image and vision computing, 2006, 24(11): 123312432 cucchiara r, grana c, tardini g, vezzani r. probabilistic people tracking for occlusion handling. in: pro