A Data-Driven Framework for Learning Dexterous Manipulation of Unknown Objects

We address the problem of developing precision, quasi-static control strategies for fingertip manipulation in robot hands. In general, analytically specifying useful object transition maps, or hand-object Jacobians, for scenarios in which there is uncertainty in some key aspect of the hand-object system is difficult or impossible. This could be in scenarios with standard fully-actuated hands where, for instance, there is not an accurate model of the contact conditions, or in scenarios with fewer control inputs than mechanical degrees of freedom (such as underactuated hands or those that are controlled by synergies or impedance controlled frameworks), since the output space is of higher dimension than the input space. In this work, we develop a method for extracting object transition maps by tracking the state of the grasp frame. We begin by modeling a compliant, underactuated hand and its mechanical properties through an energy-based approach. From this energy model, we provide controlled actuation inputs to change the state of the grasp frame. We observe the response from these actions and develop a regression map of the action-reaction pairs, where the map is subject to our intent for grasp frame movement and the regional relationship between the contacts. Once the regression model is developed, we perform within-hand planning of the grasp frame with newly introduced objects. This approach is agnostic to the global geometry of the object and is able to adapt when undesirable contact conditions, such as sliding, occur. The learning-based methodology estimates the non-linearities representative in the properties of the system. We test our framework physically on an adapted Yale Openhand Model O. By transferring the learned model from simulation to the physical hand without adaptation, we show that this energy modeling approach is robust to inaccuracies in parameter estimation. We demonstrate its efficacy in a handwriting task.