Integration of eye tracking capability into optical see-through head-mounted displays

While head-mounted display (HMD) technologies have undergone great developments since the first HMD for 3D visualization originated by Ivan Sutherland in the 1960's, HMD technologies have suffered from tradeoffs in capability and limitation. The main issues of HMDs include the tradeoff between resolution and field of view (FOV), the presence of large distortion for wide field of view designs, the inaccurate eye point representation, the conflict of accommodation and convergence, the occlusion contradiction between virtual and real objects, the challenge of highly precise registration, and often the brightness conflict with background illumination. Some of these issues impose critical affects on accuracy of visualization, user performance and safety. Among these issues, the ignorance of eye movement is a typical aspect. Most of the current HMD technology uses head pose to approximate line-of-sight, which may cause significant disparity between what you are intended to look at and what you actually look at as a result of at least +/- 20 degrees eye movement. It is highly necessary to integrate eye-tracking capability into HMDs in some demanding applications. An HMD-eyetracker integrated system would be able to display stereoscopic virtual images as a classical HMD does, and also be able to tracking the gaze direction of the user. This paper will provide a brief survey of the eye tracking technologies that are suitable for HMD integration purpose, present an approach to integrating eye tracking technology into optical see-through type of HMDs, and discuss critical issues challenging the integration. Furthermore, early engineering implementation and experimental results are published.