2.5D Cascaded Regression for Robust Facial Landmark Detection

In this paper, we propose a 2.5D Cascaded Regression approach for accurately and robustly locating facial landmarks on RGB-D data. Instead of detecting facial landmarks on texture and depth images separately, the proposed method alternately applies depth-based and texture-based regressors to compute the necessary increments to the estimated landmarks so that they are gradually moved towards their true positions. This way, depth information is better explored through close interaction with texture information, and they together improve the facial landmark detection accuracy. Moreover, thanks to the robustness of depth information to illumination variations and its capacity of capturing the deformations caused by pose and expression changes, the proposed method has good robustness to pose, illumination and expression (PIE) variations. We have extensively evaluated the effectiveness of depth information and compared the proposed method with state-of-the-art texture-based and RGB-D-based methods on three publicly accessible databases, i.e., LIDF EURECOM and Curtin-Faces. The evaluation results validate the superiority of our approach in utilizing depth information for accurately detecting facial landmarks under challenging conditions with obvious PIE variations.