MoireTracker: Continuous Camera-to-Screen 6-DoF Pose Tracking Based on Moire Pattern

In the realm of AR applications and particularly camera-to-screen interactions, camera tracking stands as a crucial technology. However, the ever-increasing demand for tracking accuracy makes it essential to explore a six-degrees of freedom (6-DoF) tracking technology with ultra-high precision to facilitate micro-motion sensing. In this paper, we propose a novel sensing method MoireTracker to achieve camera's 6-DoF pose tracking with ultra-high precision. MoireTracker outputs camera's continuous 3-DoF trajectory and 3-DoF posture changes according to the captured moir & eacute; patterns, which can be produced by the superposition of camera's Color Filter Array (CFA) and the projection of screen raster on the CFA plane. Thanks to moir & eacute; pattern's high sensitivity to 6-DoF motions, we characterize the relationship between moir & eacute; features and camera's micro pose changes, so as to realize the continuous 6-DoF pose tracking for camera with ultra-high precision. Moreover, our proposal involves a thumbnail-based method aimed at expanding the working range of MoireTracker, enabling the pervasive camera-to-screen interactions. We implement a prototype system and evaluate its performance in real-world environments. Extensive experiment results show that MoireTracker achieves the average trajectory error of 1.20 cm and the posture error of 1.07 degrees.