Fast and robust active camera relocalization in the wild for fine-grained change detection

Active camera relocalization (ACR) is an important and challenging task, whose feasibility and success highly depend on illumination consistency and convergence speed. If under varied lighting conditions in outdoor scenes, however, both the convergence and accuracy of ACR cannot be guaranteed. In this paper, we propose a fast and robust ACR scheme, namely rACR, that works well under highly varied illu-minations. To achieve robustness to lighting variations, rather than using 2D feature matching, we rely on 3D point clouds, acquired by a visual SLAM engine (VSE), to register the current and reference camera coordinate frames. We present a scale-aware point cloud matching function that is minimized by a two-stage coarse-to-fine method, i.e., fast alignment considering only geometric error at first, followed by fine-grained alignment optimizing both geometric, photometric errors and the poses of VSE key-frames. The two aligned point clouds with equalized scales help to bridge current and reference observa-tions, avoiding 2D feature matching that are sensitive to large lighting variances, and can directly generate effective camera pose adjustments. Moreover, to achieve fast convergence speed, we implement the above algorithm with a parallel scheme, which is specifically composed of an initialization procedure and three parallel threads, i.e., VSE thread, pose alignment thread, and pose adjustment thread. Extensive experiments show that, rACR has much higher robustness to lighting variations and 5x faster conver-gence rate over state-of-the-art methods, thus significantly improves its feasibility in real-world fine-grained change detection tasks in the wild.(c) 2022 Elsevier B.V. All rights reserved.