A Dynamic Region-of-Interest Vision Tracking System Applied to the Real-Time Wing Kinematic Analysis of Tethered Drosophila

Tracking objects that move fast with respect to their size is challenging because it necessitates large field of views often incompatible with the required spatial and temporal resolutions. Here, we present a novel computer vision system that overcomes this tradeoff by employing a camera with dynamic region-of-interest (ROI) capabilities, combined with an efficient predictive approach. We apply this method to extract the wing kinematics of tethered flying fruit flies in real time. At each frame, only the pixels immediately surrounding the wing are exposed, and the wing position is extracted. It is then fed to an extended Kalman filter that extracts four key parameters of the measurement time-course and, therefore, provides real-time feedback of wing motion. Using this approach, we are able to sample the wing position of both wings at 7 kHz in a 2500 pixel ROI. Our methods promise new applications that can be implemented in general purpose digital hardware for high performance tracking and process control in a broad range of applications in technology and science. Note to Practitioners-Real-time, high-speed tracking applications are subject to a bandwidth tradeoff between a large field-of-view, a high frame rate and a fine spatial resolution. Because relevant information in tracking problems is often localized in space and time, the bandwidth limitation can be partly overcome from a selective transfer and analysis of only those image data that are of momentary relevance for the tracking task at hand. Based on this approach, we present methods implemented in standard digital hardware that allow a small subset of pixels containing relevant information to be selectively exposed, transferred and analyzed in real time. A prediction model is used to perform this selection and, furthermore, to provide a parametrization of the periodic object motion to control external hardware in real time. Beyond the study of insect flight control, this paper demonstrates a novel approach to track complex and fast moving structures in real-time applications, a challenge often faced in micro and nanotechnologies.