3W-AlignNet: a Feature Alignment Framework for Person Search with Three-Way Decision Theory

Person search aims to locate and recognize a specified person from a gallery of uncropped scene images, which combines pedestrian detection and person re-identification (re-ID). Existing methods based on Faster R-CNN have been widely used to tackle the two sub-tasks jointly, but they ignore the feature misalignment problem, i.e., re-ID feature localization is not fully aligned with the detected bounding boxes (BBoxes). Due to the fine-grained property of re-ID, it is crucial to extract accurate appearance features. In addition, the granularity of BBoxes detected from gallery images is quite different, and it is defective to treat gallery boxes with different granularity as equal in estimating their similarities with the query. Three-way decision methods are fields of research on human-inspired computation. Inspired by them, we propose a three-way-based feature alignment framework (3W-AlignNet) to optimize the re-ID feature localization. The framework is implemented by iteratively generating new BBoxes and features from previous BBoxes. The three-way decision theory is applied to avoid the mismatch problem caused by increasing Intersection over Union (IoU). We further propose a Granularity Weighted Similarity (GWS) algorithm to relieve the granularity mismatch problem. Extensive experiments show that our method outperforms all other state-of-the-art end-to-end methods on two widely used person search datasets, CUHK-SYSU and PRW.