Real-time Hand-object Occlusion for Augmented Reality Using Hand Segmentation and Depth Correction

被引:0
|
作者
Wu, Yuhui [1 ]
Liu, Yue [1 ]
Wang, Jiajun [1 ]
机构
[1] Beijing Inst Technol, Sch Opt & Photon, Beijing Engn Res Ctr Mixed Real & Adv Display, Beijing, Peoples R China
基金
中国国家自然科学基金;
关键词
Human-centered computing-Human computer interaction (HCI)-Interaction paradigms Mixed/ augmented reality; Computing methodologies-Computer graphics-Image manipulation-Image processing;
D O I
10.1109/VRW58643.2023.00158
中图分类号
TP18 [人工智能理论];
学科分类号
081104 ; 0812 ; 0835 ; 1405 ;
摘要
Hand-object occlusion is crucial to enhance the realism of Augmented Reality, especially for egocentric hand-object interaction scenes. In this paper, a hand segmentation-based depth correction approach is proposed, which can help to realize real-time hand-object occlusion. We introduce a lightweight convolutional neural network to quickly obtain real hand segmentation mask. Based on the hand mask, different strategies are adopted to correct the depth data of hand and non-hand regions, which can implement hand-object occlusion and object-object occlusion simultaneously to deal with complex hand situations during interaction. The experimental results demonstrate the feasibility of our approach presenting visually appealing occlusion effects.
引用
收藏
页码:631 / 632
页数:2
相关论文
共 50 条
  • [1] HAND SEGMENTATION FOR HAND-OBJECT INTERACTION FROM DEPTH MAP
    Kang, Byeongkeun
    Tan, Kar-Han
    Jiang, Nan
    Tai, Hung-Shuo
    Tretter, Daniel
    Truong Nguyen
    [J]. 2017 IEEE GLOBAL CONFERENCE ON SIGNAL AND INFORMATION PROCESSING (GLOBALSIP 2017), 2017, : 259 - 263
  • [2] Single Depth View Based Real-Time Reconstruction of Hand-Object Interactions
    Zhang, Hao
    Zhou, Yuxiao
    Tian, Yifei
    Yong, Jun-Hai
    Xu, Feng
    [J]. ACM TRANSACTIONS ON GRAPHICS, 2021, 40 (03):
  • [3] Interaction Fusion: Real-time Reconstruction of Hand Poses and Deformable Objects in Hand-object Interactions
    Zhang, Hao
    Bo, Zi-Hao
    Yong, Jun-Hai
    Xu, Feng
    [J]. ACM TRANSACTIONS ON GRAPHICS, 2019, 38 (04):
  • [4] Hand Pose Estimation for Hand-Object Interaction Cases using Augmented Autoencoder
    Li, Shile
    Wang, Haojie
    Lee, Dongheui
    [J]. 2020 IEEE INTERNATIONAL CONFERENCE ON ROBOTICS AND AUTOMATION (ICRA), 2020, : 993 - 999
  • [5] Real-Time Hand Model Estimation from Depth Images for Wearable Augmented Reality Glasses
    Zhou, Bill
    Yu, Alex
    Menke, Joseph
    Yang, Allen Y.
    [J]. ADJUNCT PROCEEDINGS OF THE 2019 IEEE INTERNATIONAL SYMPOSIUM ON MIXED AND AUGMENTED REALITY (ISMAR-ADJUNCT 2019), 2019, : 269 - 273
  • [6] REAL-TIME 3D HAND-OBJECT POSE ESTIMATION FOR MOBILE DEVICES
    Yin, Yue
    McCarthy, Chris
    Rezazadegan, Dana
    [J]. 2021 IEEE INTERNATIONAL CONFERENCE ON IMAGE PROCESSING (ICIP), 2021, : 3288 - 3292
  • [7] Resolving hand-object occlusion for mixed reality with joint deep learning and model optimization
    Feng, Qi
    Shum, Hubert P. H.
    Morishima, Shigeo
    [J]. COMPUTER ANIMATION AND VIRTUAL WORLDS, 2020, 31 (4-5)
  • [8] Hand segmentation for augmented reality system
    Yueming, Wu
    Hanwu, He
    Tong, Ru
    Detao, Zheng
    [J]. SECOND WORKSHOP ON DIGITAL MEDIA AND ITS APPLICATION IN MUSEUM & HERITAGE, PROCEEDINGS, 2007, : 395 - 400
  • [9] Real-Time Occlusion Handling in Augmented Reality Based on an Object Tracking Approach
    Tian, Yuan
    Guan, Tao
    Wang, Cheng
    [J]. SENSORS, 2010, 10 (04) : 2885 - 2900
  • [10] The Object at Hand: Automated Editing for Mixed Reality Video Guidance from Hand-Object Interactions
    Lu, Yao
    Mayol-Cuevas, Walterio W.
    [J]. 2021 IEEE INTERNATIONAL SYMPOSIUM ON MIXED AND AUGMENTED REALITY (ISMAR 2021), 2021, : 90 - 98