Beyond Shared Autonomy: Joint Perception and Action for Human-In-The-Loop Mobile Robot Navigation Systems

In this study, we present a road map from shared to full autonomy for human-in-the-loop mobile robot navigation systems. We proposed a shared autonomy framework that incorporates human-robot joint perception and action to enhance the practicality and applicability of the mobile robot navigability. Accuracy of robotic sensing and precision of robotic action are employed as autonomous safety in the loop of human control. In shared autonomy, autonomous safety is incorporated into human-teleoperated robot control and their integration is adjusted through an online user-customizable arbitration function. Beyond the current state of the art in shared autonomy, social skills and social preferences in terms of human perception, as well as cognitive decision-making and action, are compiled into autonomous behaviors through learning from demonstration method. Autonomous behaviors exported from the trained neural networks are integrated with autonomous safety and then adjusted by user-desired control arbitration for robot autonomy. The transition of shared and full autonomy is easily managed by users, depending on specific applications. To validate the methodological approach, we implemented the framework on two mobile robot platforms to evaluate its feasibility, practicability, and reproducibility. Our experimental results showed that the shared autonomy framework was well applied to incorporating personal skills and social preferences in mobile robot navigation systems. To a certain extent, the framework plays the role of the road map guiding how to take advantage of human cognitive perception and decision and precision of robotic action in developing mobile robot navigation systems that can be deployed and applied to real-world applications.