ChatDirector: Enhancing Video Conferencing with Space-Aware Scene Rendering and Speech-Driven Layout Transition

被引：1

作者：

Qian, Xun ^{[1
,2
]}

Tan, Feitong ^{[1
]}

Zhang, Yinda ^{[1
]}

Collins, Brian Moreno ^{[3
]}

Kim, David ^{[4
]}

Olwal, Alex ^{[1
]}

Ramani, Karthik ^{[2
]}

Du, Ruofei ^{[3
]}

机构：

[1] Google Res, Mountain View, CA USA

[2] Purdue Univ, W Lafayette, IN USA

[3] Google Res, San Francisco, CA 94103 USA

[4] Google Res, Zurich, Switzerland

来源：

PROCEEDINGS OF THE 2024 CHI CONFERENCE ON HUMAN FACTORS IN COMPUTING SYTEMS, CHI 2024 | 2024年

关键词：

video conferencing; 3D portrait avatar; tele-presence; attention transition; depth map; depth estimation; machine learning; video-mediated communication; collaborative work; augmented communication; FACE-TO-FACE;

D O I：

10.1145/3613904.3642110

中图分类号：

TP18 [人工智能理论];

学科分类号：

081104 ; 0812 ; 0835 ; 1405 ;

摘要：

Remote video conferencing systems (RVCS) are widely adopted in personal and professional communication. However, they often lack the co-presence experience of in-person meetings. This is largely due to the absence of intuitive visual cues and clear spatial relationships among remote participants, which can lead to speech interruptions and loss of attention. This paper presents ChatDirector, a novel RVCS that overcomes these limitations by incorporating space-aware visual presence and speech-aware attention transition assistance. ChatDirector employs a real-time pipeline that converts participants' RGB video streams into 3D portrait avatars and renders them in a virtual 3D scene. We also contribute a decision tree algorithm that directs the avatar layouts and behaviors based on participants' speech states. We report on results from a user study (N=16) where we evaluated ChatDirector. The satisfactory algorithm performance and complimentary subject user feedback imply that ChatDirector significantly enhances communication efficacy and user engagement.

引用

页数：16