Multilevel Attention Siamese Network for Keypoint Detection in Optical and SAR Images

Optical and synthetic aperture radar (SAR) image keypoint detection is an important foundation for multimodal remote sensing image matching. The influence of nonlinear radiometric differences and geometric deformation between optical and SAR images leads to low repeatability of existing keypoint detection methods. To address the problem that existing keypoint detection methods cannot provide the required homonymous points for heterogeneous image matching, we propose a keypoint detection method [multilevel attention Siamese network for keypoint detection in optical and SAR images (SKD-Net)] and improve it in terms of both network structure and network optimization. First, we propose a multilevel attention Siamese network, which is composed of multiple convolutional modules and transformer modules with shared weights to extract common features at different levels for keypoint detection. We introduce a transformer module in the keypoint detection pipeline and fuse shallow and deep features to obtain more spatial and rich semantic information to facilitate heterogeneous image keypoint detection. Then, to ensure that the detected keypoints have more homonymous points and localization accuracy, we propose a position-consistent loss. Unlike previous loss functions, our designed position-consistent loss function takes the differences between heterogeneous image score maps into account, and it autonomously selects the optimized correct point pairs to enable the network to perform correct learning. Finally, extensive experiments show that our detection method outperforms the current state-of-the-art keypoint detection methods in terms of repeatability, localization accuracy, and matching performance. Our source code is available at https://github.com/zhangschen/SKD-Net.