行人再识别的若干问题课件

1、Person Re-identification:Recent Challenges1My Research2Human Identification & Activity Understandingq BackgroundThe whole story1Detect an event2track persons across camera view3Identify who he/she is3Human Identification & Activity Understandingq BackgroundActivityPerson Re-identificationFaceRecogni

2、tionPerson Re-identificationWhat ishedoing?Matching, TrackingCamera NetworkUnderstandingDetecting target objects (cars,pedestrian, bags etc.)5Person Re-identification6Recent Development & QuestionPose-guided, Local, Attention-based, GAN-based, a ppt: /5VPtcZaqWhat should we do? I would guess we will

3、 soon have 99%matching rate this year or early next year on benchmarksqHave we already solved it?q7My Todays FocusTell less about performanceqAim to tell something of my understandingabout Re-IDq8Person Re-identification: Challenges9Person Re-identification: Challengesq Some Main VariationsView Ligh

4、ting Occlusion Low Resolution Clothing Change101. Connection with Cross Domain?11Person Re-ID vs. Cross-ModalityView Biasq12Asymmetric Metric for Re-IDLearning universal featuretransformationLearning view-specificfeature transformation13Asymmetric Metric for Re-IDLearn different featuretransformatio

5、n for differentcamera viewsPseudometricNon-negativity SymmetryTriangle InequalityCoincidence14Asymmetric Metric for Re-IDq Re-ID Reformulation by AugmentationView-specifictransformationYingcong Chen, Xiatian Zhu, Wei-Shi Zheng*, and Jian-Huang Lai. Person Re-Identificationby Camera Correlation Aware

6、 Feature Augmentation. IEEE Trans. on Pattern Analysis andMachine Intelligence (PAMI), 2017.15Asymmetric Metric for Re-IDq Re-ID Reformulation by AugmentationNot able to measure the relationshipbetween different view-specifictransformation matricesView-specifictransformationDo not constraint the dis

7、crepancybetween feature transformation acrossview:CoincidenceYingcong Chen, Xiatian Zhu, Wei-Shi Zheng*, and Jian-Huang Lai. Person Re-Identificationby Camera Correlation Aware Feature Augmentation. IEEE Trans. on Pattern Analysis andMachine Intelligence (PAMI), 2017.16Asymmetric Metric for Re-IDAda

8、ptive feature augmentationqgeneralisedcontrol thediscrepancyBetweenfa and fbYingcong Chen, Xiatian Zhu, Wei-Shi Zheng*, and Jian-Huang Lai. Person Re-Identificationby Camera Correlation Aware Feature Augmentation. IEEE Trans. on Pattern Analysis andMachine Intelligence (PAMI), 2017.17Asymmetric Metr

9、ic for Re-IDLearning:qCamera coRrelation Aware Feature augmenTation (CRAFT)Generalize any symmetric metric learning models to asymmetricones: e.g. MFA18Asymmetric Metric for Re-IDLearning:qCamera coRrelation Aware Feature augmenTation (CRAFT)Camera ViewDiscrepancyRegularization:ReduceCoincidenceBreg

10、man discrepancy of a projection19Asymmetric Metric for Re-IDA frameworkqto extractdomain-genericand more viewinvariant personfeatures20Asymmetric Metric for Re-IDEvaluation: augmentation or not augmentation?qEvaluation: augmentation vs. domain adaptationqqEvaluation: whether using Camera View Discre

11、pancy21Does the Asymmetric Metric Modelling Workfor other setting: unsupervised, semi-supervised, .22Asymmetric Metric for Re-ID: UnsupervisedUnsupervised Learningqo Clustering-based Asymmetric MEtric Learning(CAMEL)Hongxing Yu, Ancong Wu, Wei-Shi Zheng*. -Learning for Unsupervised Person Re-identif

12、ication. In IEEE Conf. on ComputerVision (ICCV), 2017.23Asymmetric Metric for Re-ID: UnsupervisedUnsupervised Learningq24Hash Re-ID for Fast SearchFAST Re-ID on Numbers of Camerasqo Learning view-specific hash code for each cameraXiatian Zhu, Botong Wu, Dongcheng Huang, Wei-Shi Zheng*(PI)Identificat

13、ion. IEEE Transactions on Image Processing, 2017. Fast Open-World Person Re-Wei-Shi Zheng, Shaogang Gong, and Tao Xiang. Towards Open-World Person Re-Identificationby One-Shot Group-based Verification. IEEE Transactions on Pattern Analysis and MachineIntelligence (PAMI), vol. 38, no. 3, pp. 591-606,

14、 2016.25Hash Re-ID for Fast SearchIdea of the FormulationqCross-view IdentityVerification RegularisationCross-view IdentityCorrelation HashingView Context DiscrepancyRegularisation26Hash Re-ID for Fast SearchFAST Searchqo Comparison to other related Hashing functions27Hash Re-ID for Fast SearchFAST

15、Searchqo When using more powerful features?282. How to match heterogeneousperson images across camera views?29Person Re-ID vs. Cross-ModalityMatching between Heterogeneous Imagesq30RGB-Infrared Re-IDCross-Modality Learning: RGB-IR Re-IDqo Deep zero-paddingAncong Wu, Wei-Shi Zheng*(PI), Hongxing Yu,

16、Shaogang Gong, Jianhuang Lai. RGB-InfraredCross-Modality Person Re-Identification. In IEEE Conf. on Computer Vision (ICCV), 2017.31RGB-Infrared Re-IDCross-Modality Learning: RGB-IR Re-IDq32RGB-Infrared Re-IDCross-Modality Learning: RGB-IR Re-IDq33RGB-Infrared Re-IDCross-Modality Learning: RGB-IR Re-

17、IDqo SYSU RGB-IR Re-ID Dataset34When the input is not image?35Attribute-Image Person Re-IDMatch person images with specific attributedescriptions in surveillance environment.qZhou Yin, Wei-Shi Zheng*(PI), et al. Adversarial Attribute-Image Person Re-identification, IJCAI 201836Attribute-Image Person

18、 Re-IDIntuitively, when we hold some attribute description in mind, e.g.,qq“carrying backpack”, we generate an obscure and vagueimagination on how a backpack may look like, which we refer to asa concept.We model this generation process and match the generatedconcepts with image perceptions.37Attribu

19、te-Image Person Re-IDImage Concept Extraction loss : Our model learns a semanticallyqq!discriminative structure of low-level person images.Semantic Consistency Constraint + Adversary loss : Our model!#$%generates the corresponding aligned image-analogous concept forhigh-level attribute.38Attribute-I

20、mage Person Re-IDOur model:Outperforms traditional cross modality retrieval methods (DeepCCAE, DeepCCA, 2WayNet,CMCE).qOutperforms classical pedestrian attribute recognition model (DeepMAR).Outperforms other variants of our model, which also generate homogenous distributions undersemantic consistenc

21、y regularization for the two modalities (MMD, DeepCoral).qq39Attribute-Image Person Re-IDWrong samples40Attribute-Image Person Re-IDEffects of different generation strategies:Generation from attributes to image is better than generation fromqqimage to attributes. (A2Img vs. Img2A):Estimating the man

22、ifold of images from the training data is more reliable thanestimating that of attributesoGeneration in feature space is better than generation in real imagespace. (A2Img vs. Real Images):Generating real pedestrian image is difficult. Generating noisy low-level imagesand then eliminating these noise

23、 to extract discriminative concepts is notnecessaryo41When dressing differently?42Depth Re-IDSomething to seeqIlluminationchangeClotheschangeIn these cases, appearance cues are not reliable.43Depth Re-IDDepth descriptorsq Within-patch Covariance Between-patch Covariance Eigen-depth featureEigen-dept

24、h feature is rotation invariant.44Depth Re-IDMetricqxixjOExtracting Eigen-depth feature converts covariance matrices onRiemannian manifold to feature vectors in Euclidean space.45Depth Re-ID46Depth Re-IDTransferring Depthq()Ancong Wu, Wei-Shi Zheng*(PI), and Jian-HuangLai. Robust Depth-based Person

25、Re-identification.IEEE Transactions on Image Processing, 2017Depth Re-ID483. Low-resolutionPerson Re-identificationVaryingResolutionsCamera ACamera B49Low-resolution Re-ID50Low-resolution Re-IDLow-resolution Re-IDqo JUDEA : joint multi-scale discriminant componentanalysisXiang Li, Wei-Shi Zheng*, Xi

26、aojuan Wang, Tao Xiang, Shaogang Gong. Multi-scale(PI)Learning for Low-resolution Person Re-identification. IEEE Conf. on Computer Vision (ICCV),2015.51Low-resolution Re-IDq Super- resolution and Identity joiNt learninG(SING)Jiening Jiao, Wei-Shi Zheng*(PI), Ancong Wu, Xiatian Zhu, and Shaogang Gong

27、. Deep Low-resolution Person Re-identification. AAAI 201852Low-resolution Re-ID53Low-resolution Re-IDResultsq54Low-resolution Re-IDResultsq55More 56Cross-set Re-IDGalleryProbeLabelling images across camera views is costly57Cross-scenario Re-IDTransferring between setsAnqAsymmetricMulti-taskModelling

28、Xiaojuan Wang, Wei-Shi Zheng*(PI), Xiang Li, and Jianguo Zhang. Cross-scenario Transfer Person Re-identification. IEEE Transactions on Circuits and Systems for Video Technology, vol. 26, no. 8, pp.1447-1460, 2016.58Partial Re-ID59Partial Re-IDAnnotating PartialPart by Operatoror Detecting itautomati

29、callyLocal-to-localMatchingMatchingFusionWei-Shi Zheng, Xiang Li, Tao Xiang,Shengcai Liao, JianHuang Lai, ShaogangGong. Partial Person Re-identification.ICCV, 2015.Global-to-localMatching60Partial Re-IDExample of partial person matching61One-Shot Open-World Group-based Re-idMotivationqOpe world personr identification setting1) A large amount of non-targetimposters captured alongwith the target people on thewatch list.2) Their images will also appearin the probe set and some ofthem will look visually similarto the target peopleWei-Shi Zheng, Shaogang Go