SMARTTALK: A Learning-based Framework for Natural Human-Robot Interaction

This paper presents a learning-based framework named SMARTTALK for natural-language human-robot interaction (HRI). The primary goal of this framework is to enable non-expert users to control and program a mobile robot using natural language commands. SMARTTALK is modality-agnostic, and is capable of integrating with both speech and non-speech (e.g., gesture-based) communication. Initially, robots using this mechanism are equipped with a limited vocabulary of primitive commands and functionality; however, through extended use and interaction, the robots are able to learn new commands and adapt to user's behaviors and habits. This makes the proposed framework highly desirable for long-term deployment in a variety of HRI tasks. We present the design of this framework and experimental data on a number of realistic scenarios to evaluate its performance. A qualitative experiment on a robotic platform is also presented.