Cooperative Eye Tracking and Localization Based on Corneal Imaging

Corneal imaging has been studied as an eye-tracking method, and the focused object and point-of-gaze are estimated by extracting scene information reflected on the corneal surface. However, when an eye tracker is regarded as a daily-use device, it is an essential function to estimate its self-position. In our previous research, we attempted self-position estimation using interior illumination patterns reflected on the cornea. However, the accuracy of localization was not sufficient; thus, we focused on convolutional neural network developed remarkably and the inertial measurement unit (IMU) as additional improvements. In this study, we propose to improve the accuracy of localization by integrating SE-ResNet and IMU with Monte Carlo localization. Data collection and annotation become a problem when deep neural network is employed. To address this problem, three-dimensional (3D) environmental map was reconstructed, and SE-ResNet was trained using virtual images generated using the 3D environmental map. We conducted comparative experiments and confirmed the feasibility of our proposed method.