Assembly Design and Evaluation based on Bare-Hand Interaction in an Augmented Reality Environment

Augmented Reality (AR) is an increasingly important application domain for computer graphics and user interface design. In this paper, the authors propose an AR interface for assembly design and evaluation. The methodologies to establish an AR assembly environment and the architecture of the proposed system are presented. A key novelty of this research is the use of bare hands for human-computer interaction in the AR assembly environment. An effective hand segmentation method based on a Restricted Coulomb Energy (RCE) neural network is developed. Experimental results show that the proposed hand segmentation method is effective and robust and the finger tracking algorithm runs in real time under a wide range of lighting conditions. An experiment using two fingertips to control virtual objects to simulate assembly in a 2D space is conducted.