Volumetric Video Reconstruction and Communications: Toward a New Era of Interactive and Immersive Social Virtual Reality (VR) Experiences

Recent advances in volumetric video technologies are opening the door to new interactive and immersive experiences. This paper presents novel technological enablers to provide realistic and volumetric reconstructions of real-world elements. On the one hand, novel Neural Radiance Field (NeRF) based methods are proposed to increase the quality of 3D scenes and objects reconstructions, with improved smoothness of the surfaces. On the other hand, a real-time volumetric video pipeline is proposed to overcome limitations in terms of visual quality and resources usage (bandwidth and processing) of state-of-the-art solutions adopted in the holographic communications domain. Preliminary results not only prove the benefits of such enablers when compared to state-of-the-art solutions, but the potential of their joint integration to provide new compelling and engaging interactive virtual experiences.