Pose and Skeleton-aware Neural IK for Pose and Motion Editing

被引:0
|
作者
Agrawal, Dhruv [1 ,2 ]
Guay, Martin [2 ]
Buhmann, Jakob [2 ]
Borer, Dominik [2 ]
Sumner, Robert W. [1 ,2 ]
机构
[1] Swiss Fed Inst Technol, Zurich, Switzerland
[2] DisneyRes Studios, Zurich, Switzerland
关键词
skeletal networks; pose authoring; learned inverse kinematics; 3D animation;
D O I
10.1145/3610548.3618217
中图分类号
TP18 [人工智能理论];
学科分类号
081104 ; 0812 ; 0835 ; 1405 ;
摘要
Posing a 3D character for film or game is an iterative and laborious process where many control handles (e.g. joints) need to be manipulated to achieve a compelling result. Neural Inverse Kinematics (IK) is a new type of IK that enables sparse control over a 3D character pose, and leverages full body correlations to complete the un-manipulated joints of the body. While neural IK is promising, current methods are not designed to preserve previous edits in posing workflows. Current models generate a single pose from the handles only-regardless of what was there previously-making it difficult to preserve any variations and hindering tasks such as pose and motion editing. In this paper, we introduce SKEL-IK, a novel architecture and training scheme that is conditioned on a base pose, and designed to flow information directly onto the skeletal graph structure, such that hard constraints can be enforced by blocking information flows at certain joints. As a result, we are able to satisfy both hard and soft constraints, as well as preserve un-manipulated parts of the body when desired. Finally, by controlling the base pose in different ways, we demonstrate the ability of our model to perform tasks such as generating variations and quickly editing poses and motions; with less erosion of the base poses compared to the current state-of-the-art.
引用
收藏
页数:10
相关论文
共 50 条
  • [31] Learning Pose-Aware Models for Pose-Invariant Face Recognition in the Wild
    Masi, Iacopo
    Chang, Feng-Ju
    Choi, Jongmoo
    Harel, Shai
    Kim, Jungyeon
    Kim, Kanggeon
    Leksut, Jatuporn
    Rawls, Stephen
    Wu, Yue
    Hassner, Tal
    AbdAlmageed, Wael
    Medioni, Gerard
    Morency, Louis-Philippe
    Natarajan, Prem
    Nevatia, Ram
    IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, 2019, 41 (02) : 379 - 393
  • [32] Pose-Aware Disentangled Multiscale Transformer for Pose Guided Person Image Generation
    Shibasaki, Kei
    Ikehara, Masaaki
    IEEE ACCESS, 2023, 11 : 146054 - 146064
  • [33] Efficient Pose: Efficient human pose estimation with neural architecture search
    Wenqiang Zhang
    Jiemin Fang
    Xinggang Wang
    Wenyu Liu
    Computational Visual Media, 2021, 7 (03) : 335 - 347
  • [34] Mutually reinforcing motion-pose framework for pose invariant action recognition
    Ramanathan, Manoj
    Yau, Wei-Yun
    Thalmann, Nadia Magnenat
    Teoh, Eam Khwang
    INTERNATIONAL JOURNAL OF BIOMETRICS, 2019, 11 (02) : 113 - 147
  • [35] Structure-Preserving Instance Segmentation via Skeleton-Aware Distance Transform
    Lin, Zudi
    Wei, Donglai
    Gupta, Aarush
    Liu, Xingyu
    Sun, Deqing
    Pfister, Hanspeter
    MEDICAL IMAGE COMPUTING AND COMPUTER ASSISTED INTERVENTION, MICCAI 2023, PT III, 2023, 14222 : 529 - 539
  • [36] POSE GUIDED PERSON IMAGE GENERATION BASED ON POSE SKELETON SEQUENCE AND 3D CONVOLUTION
    Zhao, Wenbin
    Xie, Qing
    Ma, Yanchun
    Liu, Yongjian
    Xiong, Shengwu
    2020 IEEE INTERNATIONAL CONFERENCE ON IMAGE PROCESSING (ICIP), 2020, : 1561 - 1565
  • [37] Pose2Drone: A Skeleton-Pose-based Framework for Human-Drone Interaction
    Marinov, Zdravko
    Vasileva, Stanka
    Wang, Qing
    Seibold, Constantin
    Zhang, Jiaming
    Stiefelhagen, Rainer
    29TH EUROPEAN SIGNAL PROCESSING CONFERENCE (EUSIPCO 2021), 2021, : 776 - 780
  • [38] Observing pose and motion through contact
    Jia, YB
    Erdmann, M
    1998 IEEE INTERNATIONAL CONFERENCE ON ROBOTICS AND AUTOMATION, VOLS 1-4, 1998, : 723 - 729
  • [39] Pose controlled physically based motion
    Fattal, Raanan
    Lischinski, Dam
    COMPUTER GRAPHICS FORUM, 2006, 25 (04) : 777 - 787
  • [40] Pose-Free Neural Radiance Fields via Implicit Pose Regularization
    Zhang, Jiahui
    Zhan, Fangneng
    Yu, Yingchen
    Liu, Kunhao
    Wu, Rongliang
    Zhang, Xiaoqin
    Shao, Ling
    Lu, Shijian
    2023 IEEE/CVF INTERNATIONAL CONFERENCE ON COMPUTER VISION, ICCV, 2023, : 3511 - 3520