Annotation scaffolds for manipulating articulated objects

We present and evaluate an approach for human-in-the-loop specification of robot–object interactions. Our method is based on the idea of model annotation: the addition of simple cues to an underlying object model to delineate a complex task. The goal is to explore simplified CAD-like interfaces to permit novice users to describe and annotate manipulation tasks that are then carried out by a robot. The constructed models meet precision requirements for modeling a variety of joint types and kinematic chains, and can be re-used after their initial design. The approach is contrasted with teleoperation and tested with a user study. We found that untrained users can create object models whose structure can be readily used to initialize joint constraints to guide the user in designing successful end-effector trajectories. We see this approach as an alternative to direct teleoperation for cases where it is more natural or practical to store the action sequence with the object as the reference frame. The approach was evaluated using the PR2 robot platform.