Diver Gesture Recognition using Deep Learning for Underwater Human-Robot Interaction

This paper presents a diver gesture recognition method for autonomous underwater vehicles (AUVs) to facilitate human-robot collaborative tasks. While previous methods of underwater human-robot communication required the use of expensive and bulky keyboard or joystick controls, hand gestures are becoming more popular for underwater reprogramming of AUVs because they are easier to use, faster, and cost effective. However, most of the existing datasets for the hand gesture recognition problem, were either based on unrealistic environments such as swimming pools or utilized ineffective sensor configurations. Recently, Cognitive Autonomous Driving Buddy (CADDY) dataset was released to the public which overcomes the limitations of the existing datasets. It contains the images of different diver gestures in several different and realistic underwater environments, including a set of true negatives such as divers with improper gestures or no gestures. To the best of our knowledge, this dataset has not yet been tested for gesture classification; as such, this paper presents the first benchmark results for efficient underwater human-robot interaction. In the proposed framework, a deep transfer learning approach is utilized to achieve high correct classification rate (CCR) of up to 95%. The classifier is constructed in relatively short amount of training time, while it is suitable for real-time underwater diver gesture recognition.