Real-Time Accurate 3D Head Tracking and Pose Estimation with Consumer RGB-D Cameras

被引:0
|
作者
David Joseph Tan
Federico Tombari
Nassir Navab
机构
[1] Technische Universität München,
[2] Università di Bologna,undefined
来源
关键词
3d temporal tracking; Head pose estimation; Real time; Random forests; Multi camera;
D O I
暂无
中图分类号
学科分类号
摘要
We demonstrate how 3D head tracking and pose estimation can be effectively and efficiently achieved from noisy RGB-D sequences. Our proposal leverages on a random forest framework, designed to regress the 3D head pose at every frame in a temporal tracking manner. One peculiarity of the algorithm is that it exploits together (1) a generic training dataset of 3D head models, which is learned once offline; and, (2) an online refinement with subject-specific 3D data, which aims for the tracker to withstand slight facial deformations and to adapt its forest to the specific characteristics of an individual subject. The combination of these works allows our algorithm to be robust even under extreme poses, where the user’s face is no longer visible on the image. Finally, we also propose another solution that utilizes a multi-camera system such that the data simultaneously acquired from multiple RGB-D sensors helps the tracker to handle challenging conditions that affect a subset of the cameras. Notably, the proposed multi-camera frameworks yields a real-time performance of approximately 8 ms per frame given six cameras and one CPU core, and scales up linearly to 30 fps with 25 cameras.
引用
收藏
页码:158 / 183
页数:25
相关论文
共 50 条
  • [1] Real-Time Accurate 3D Head Tracking and Pose Estimation with Consumer RGB-D Cameras
    Tan, David Joseph
    Tombari, Federico
    Navab, Nassir
    [J]. INTERNATIONAL JOURNAL OF COMPUTER VISION, 2018, 126 (2-4) : 158 - 183
  • [2] GEOMETRICAL 3D RECONSTRUCTION USING REAL-TIME RGB-D CAMERAS
    Penelle, B.
    Schenkel, A.
    Warzee, N.
    [J]. INTERNATIONAL CONFERENCE ON 3D IMAGING 2011 (IC3D 2011), 2011,
  • [3] Real-time tracking of 3D elastic objects with an RGB-D sensor
    Petit, Antoine
    Lippiello, Vincenzo
    Siciliano, Bruno
    [J]. 2015 IEEE/RSJ INTERNATIONAL CONFERENCE ON INTELLIGENT ROBOTS AND SYSTEMS (IROS), 2015, : 3914 - 3921
  • [4] Real-time head tracking and 3D pose estimation from range data
    Malassiotis, S
    Strintzis, MG
    [J]. 2003 INTERNATIONAL CONFERENCE ON IMAGE PROCESSING, VOL 2, PROCEEDINGS, 2003, : 859 - 862
  • [5] Head Pose Free 3D Gaze Estimation Using RGB-D Camera
    Kacete, Amine
    Seguier, Renaud
    Collobert, Michel
    Royan, Jerome
    [J]. EIGHTH INTERNATIONAL CONFERENCE ON GRAPHIC AND IMAGE PROCESSING (ICGIP 2016), 2017, 10225
  • [6] Real-time Pose Estimation of Rigid Objects using RGB-D Imagery
    Asif, Umar
    Bennamoun, Mohammed
    Sohel, Ferdous
    [J]. PROCEEDINGS OF THE 2013 IEEE 8TH CONFERENCE ON INDUSTRIAL ELECTRONICS AND APPLICATIONS (ICIEA), 2013, : 1692 - 1699
  • [7] A Real-Time Infrared Stereo Matching Algorithm for RGB-D Cameras' Indoor 3D Perception
    Zhong, Jiageng
    Li, Ming
    Liao, Xuan
    Qin, Jiangying
    [J]. ISPRS INTERNATIONAL JOURNAL OF GEO-INFORMATION, 2020, 9 (08)
  • [8] Real-Time Passable Area Segmentation With Consumer RGB-D Cameras for the Visually Impaired
    Zou, Wenbin
    Hua, Guoguang
    Zhuang, Yue
    Tian, Shishun
    [J]. IEEE TRANSACTIONS ON INSTRUMENTATION AND MEASUREMENT, 2023, 72
  • [9] Feature-based RGB-D camera pose optimization for real-time 3D reconstruction
    Wang C.
    Guo X.
    [J]. Guo, Xiaohu (xguo@utdallas.edu), 1600, Tsinghua University Press (03): : 95 - 106
  • [10] Feature-based RGB-D camera pose optimization for real-time 3D reconstruction
    Chao Wang
    Xiaohu Guo
    [J]. Computational Visual Media, 2017, 3 (02) : 95 - 106