Real-Time Hole-Filling in Mobile Augmented Reality Gaming: A Novel Algorithm to Overcome Depth Sensor Limitations

In the realm of Augmented Reality (AR) within mobile gaming, the planes recognized by depth sensors delineate the space for content implementation, thereby constraining the scope of representation. Errors in these recognized planes may inhibit the progression of mobile AR. A method is explored that utilizes 'Meshing' facilitated by Unity, generating meshes corresponding to physical space and enabling expansion of content space beyond mere planes. This approach, although promising, is contingent on the depth sensor, leading to the creation of holes beyond the sensor's reach. These holes present a critical issue, allowing game objects to escape. To address this challenge, an algorithm is proposed that consists of two main components: 'Hole-Finding' and 'Hole-Filling'. In 'Hole-Finding', real holes are identified by the calculation of the direction of each loop. Subsequently, 'Hole-Filling' computes the centroid-vertex of each hole and employs it for the hole-filling process. A real-time hole-filling performance with only a 7 mu sec degradation was observed, heralding a significant step towards mitigating this problem within AR content. This investigation contributes a novel solution to a crucial technical obstacle, thereby enhancing the functionality and potential of AR in mobile gaming.