Autonomous Robot Navigation with Self-learning for Collision Avoidance with Randomly Moving Obstacles

This paper develops a hierarchical controller to avoid randomly moving obstacles in autonomous navigation of a robot. The developed method consists of two parts: a high-level Q-Iearning controller for choosing an optimal plan for navigation and a low-level, appearance-based visual servo (ABVS) controller for motion execution. The use of robot learning ability in collision avoidance is a novel feature, in a combined system framework of planning and visual servo control. The developed approach takes advantage of the on-board camera of robot whose finite field of view is naturally suitable for the Q-Iearning algorithm. Because of the Q-Iearning controller, knowledge of obstacle movement and a control law for the ABVS controller are not needed. This is a significant computational advantage. The method is implemented in a simulation system of robot navigation. The results show that Q-Iearning, which is a method of reinforcement learning, successfully converges to an optimal strategy for the robot to establish a proper motion plan.