Content-augmented feature pyramid network with light linear spatial transformers for object detection

As one of the prevalent components, feature pyramid network (FPN) is widely used in current object detection models for improving multi-scale object detection performance. However, its feature fusion mode is still in a misaligned and local manner, thus limiting the representation power. To address the inherited defects of FPN, a novel architecture termed content-augmented feature pyramid network (CA-FPN) is proposed in this paper. Firstly, a global content extraction module (GCEM) is proposed to extract multi-scale context information. Secondly, lightweight linear spatial Transformer connections are added in the top-down pathway to augment each feature map with multi-scale features, where a linearized approximate self-attention function is designed for reducing model complexity. By means of the self-attention mechanism in Transformer, it is no longer needed to align feature maps during feature fusion, thus solving the misaligned defect. By setting the query scope to the entire feature map, the local defect can also be solved. Extensive experiments on COCO and PASCAL VOC datasets demonstrated that the CA-FPN outperforms other FPN-based detectors without bells and whistles and is robust in different settings.