Pose-Guided Modality-Invariant Feature Alignment for Visible–Infrared Object Re-Identification

Visible-infrared object re-identification (VI-ReID) is the task of identifying the same object across cameras with different modalities. It is more challenging than the common ReID task since the complex intra-class interference and large modality divergence. Existing works mainly tend to adopt the instance-level or modality-center-level metric learning to extract the global features or rigid local features without considering modality alignment comprehensively, leading to a limited capacity of modality-invariant feature learning. In response to these challenges, a new approach for VI-ReID called Pose-guided Modality-invariant Feature Alignment (PMFA) is proposed, which can simultaneously eliminate the intra-class interference in each modality and bridge the gap between different modalities. Specifically, a Pose-guided Feature Enhancement (PFE) block is presented to enhance feature discrimination by introducing person key part features, which can eliminate the intra-class interference effectively and guide the ReID model to learn cross-modality consistent features of the same person at the same time. Besides, a Modal Feature Alignment Learning (MFAL) method is proposed to address the substantial modality differences from two perspectives,i.e., aligning feature distributions and hierarchical aggregation. The design not only narrows the modality gap from an explicit distribution discrepancy perspective but also comprehensively considers both intra-modality and inter-modality constraints. Comprehensive experiments conducted on two challenging datasets demonstrate the superiority of the PMFA approach compared to the latest state-of-the-art methods. Particularly on the SYSU-MM01 dataset, the PMFA approach can achieve 74.22% for Rank-1 accuracy and 70.27% for mAP score, respectively. The source code is available at https://github.com/syq2021/PMFA.