Human Motion Reconstruction and Synthesis of Human Skills

Reconstructing human motion dynamics in real-time is a challenging problem since it requires accurate motion sensing, subject specific models, and efficient reconstruction algorithms. A promising approach is to construct accurate human models, and control them to behave the same way the subject does. Here, we demonstrate that the whole-body control approach can efficiently reconstruct a subject's motion dynamics in real world task-space when given a scaled model and marker based motion capture data. We scaled a biomechanically realistic musculoskeletal model to a subject, captured motion with suitably placed markers, and used an operational space controller to directly track the motion of the markers with the model. Our controller tracked the positions, velocities, and accelerations of many markers in parallel by assigning them to tasks with different priority levels based on how free their parent limbs were. We executed lower priority marker tracking tasks in the successive null spaces of the higher priority tasks to resolve their interdependencies. The controller accurately reproduced the subject's full body dynamics while executing a throwing motion in near real time. Its reconstruction closely matched the marker data, and its performance was consistent for the entire motion. Our findings suggest that the direct marker tracking approach is an attractive tool to reconstruct and synthesize the dynamic motion of humans and other complex articulated body systems in a computationally efficient manner.