Gesture-Based Control of Autonomous UAVs

Unmanned Aerial Vehicles (UAVs) have been traditionally controlled via remote control or by software, which require skill using the remote or expert programming skills. Our goal is to develop a natural mode of directing a drone's actions, akin to the forms of expression one finds between a person and a pet and hence accessible to almost any person without specialized training or expertise in using electronic gadgets. We build on prior work on analyzing video streams to use the video from the drone's on-board camera to enable gesture-based control. Our approach uses a pre-trained convolutional neural network for pose extraction, Haar cascades to identify regions of interest within the UAV's field of view, and a finite state machine to select the drone's action.