DaCFN: divide-and-conquer fusion network for RGB-T object detection

Thermal images could help visual images to improve object detection performance under low illumination. On the other hand, the complementary fusion of visual and thermal features can be challenging. In RGB-T object detection, the two-stream network structure has been widely used, in which addition operation and concatenation operation are utilized to merge feature maps. However, the addition compacts two-stream feature with inevitable distortion, while direct concatenation may bring redundancy to features. In this paper, we show that the addition operation is more suitable for common features from RGB and thermal, while the concatenation operation is more suitable for specific features unique to RGB or thermal. Then we take the divide-and-conquer strategy to propose an RGB-T detector named Divide-and-Conquer Fusion Network (DaCFN), which divides RGB and thermal features into common and specific ones and applies category-customized operations to them. Specifically, we design the Partial Coupling Net Block (PCNB), in which common features are extracted by coupled parameters and specific features by independent ones. Then the Selective Common Addition (SCA) and the Independent Specific Concatenation (ISC) are designed to fuse common and specific features, respectively. Experiments on FLIR and KAIST datasets demonstrate that our approach achieves high accuracy with high speed against other state-of-the-art RGB-T detectors.