Pseudo-Multispectral Pedestrian Detection with Deep Thermal Feature Guidance

With complementary multi-modal information (i.e. visible and thermal), multispectral pedestrian detection is essential for around-the-clock applications, such as autonomous driving, video surveillance, and vicinagearth security. Despite its broad applications, the requirements for expensive thermal device and multi-sensor alignment limit the utilization in real-world applications. In this paper, we propose a pseudo-multispectral pedestrian detection (called PseudoMPD) method, which employs the gray image converted from the RGB image to replace the real thermal image, and learns the pseudo-thermal feature through deep thermal feature guidance (TFG). To achieve this goal, we first introduce an image base-detail decomposition (IBD) module to decompose image information into base and detail parts. Afterwards, we design a base-detail hierarchical feature fusion (BHFF) module to deeply exploit the information between these two parts, and employ a TFG module to guide pseudo-thermal base and detail feature learning. As a result, our proposed method does not require the real thermal image during inference. The comprehensive experiments are performed on two public multispectral pedestrian datasets. The experimental results demonstrate the effectiveness of our proposed method.