ClickBAIT: Click-based Accelerated Incremental Training of Convolutional Neural Networks

Training deep learning models in real-time, with a human in-the-loop, could allow them to be adjusted and adapted on-the-fly, as demanded by the mission at hand. For instance, during a tracking and surveillance operation, an unmanned aerial system (UAS) operator spots the subject, and quickly trains and distributes an object-specific recognition model to other drones and cameras while the situation is unfolding. We call this type of real-time training Time-ordered Online Training (ToOT). In this work, we present ClickBAIT, a framework for performing ToOT in real-time. ClickBAIT reduces the human effort required to perform training in real-time by reducing annotations to single clicks, and employing object tracking to produce additional training events. We implement ClickBAIT for both an image classifier and object detector, and show that it improves the training benefit of clicks from 3 to 7 times on representative video sequences.