Online 3D Reconstruction and 6-DoF Pose Estimation for RGB-D Sensors

In this paper, we propose an approach to Simultaneous Localization and Mapping (SLAM) for RGB-D sensors. Our system computes 6-DoF pose and sparse feature map of the environment. We propose a novel keyframe selection scheme based on the Fisher information, and new loop closing method that utilizes feature-to-landmark correspondences inspired by image-based localization. As a result, the system effectively mitigates drift that is frequently observed in visual odometry system. Our approach gives lowest relative pose error amongst any other approaches tested on public benchmark dataset. A set of 3D reconstruction results on publicly available RGB-D videos are presented.