Sparse depth densification for monocular depth estimation

Now the dense depth prediction by single image and a few sparse depth measurements has attracted more and more attention because it provides a low-cost and efficient solution for estimating high-quality depth information. But the current existing methods for the field only take sparse depth as an independent dimension, and the relationship between sparse depth and image itself is always ignored, which undoubtedly limits the improvement of prediction accuracy. For solving the problem, in this paper, a sparse depth densification method is proposed to fully mine the relationship between sparse depth and image for achieving more accurate depth estimation. Based on a priori that the object areas of same category have similar depth values, a Depth Densification Map (DDM) is constructed by the segmentation label obtained from unsupervised image segmentation and sparse depth to realize sparse depth densification. Meantime, considering the potential error of DDM, a Depth Error Map (DEM) is designed to further correct DDM. Then, we use the idea of multi-scale fusion to build a depth estimation network. Finally, the proposed maps are combined with single image as the input of the network, and used to carry out the actual training and testing. Extensive experiments on NYU Depth v2 and Make3D datasets demonstrate the superiority of our proposed approach. Our code is available at https://github.com/Jennifer108/Sparse-Depth-Densification. https://github.com/Jennifer108/Sparse-Depth-Densification.