Registration and occlusion handling based on the FAST ICP-ORB method for augmented reality systems

Spatial position consistency and occlusion consistency are two important problems in augmented reality systems. In this paper, we proposed a novel method that can address the registration problem and occlusion problem simultaneously by using an RGB-D camera. First, to solve the image alignment errors caused by the imaging mode of the RGB-D camera, we developed a depth map inpainting method that combines the FMM and RGB-D information. Second, we established an automatic method to judge the close-range mode based on the depth histogram to solve the registration failure problem caused by hardware limitations. In the close-range mode, the registration method combining the fast ICP and ORB was adopted to calculate the camera pose. Third, we developed an occlusion handling method based on the geometric analysis of the scene. Several experiments were performed to validate the performance of the proposed method. The experimental results indicate that our method can obtain stable and accurate registration and occlusion handling results in both the close-range and non-close-range modes. Moreover, the mutual occlusion problem can be handled effectively, and the proposed method can satisfy the real-time requirements of augmented reality systems.