An Intrinsic Reward for Affordance Exploration

In this paper, we present preliminary results demonstrating how a robot can learn environmental affordances in terms of the features that predict successful control and interaction. We extend previous work in which we proposed a learning framework that allows a robot to develop a series of hierarchical, closed-loop manipulation behaviors. Here, we examine a complementary process where the robot builds probabilistic models about the conditions under which these behaviors are likely to succeed. To accomplish this, we present an intrinsic reward function that directs the robot's exploratory behavior towards gaining confidence in these models. We demonstrate how this single intrinsic motivator can lead to artifacts of behavior such as "novelty," "habituation," and "surprise." We present results using the bimanual robot Dexter, and explore these results further in simulation.