Model-Driven Compression for Digital Human Using Multi-Granularity Representations

With the popularity of the "metaverse", the applications of virtual digital humans are emerging in entertainment and communication fields, etc, where effective digital human compression schemes are urgently needed to process huge amounts of data. Most of the existing coding methods focus on reducing spatial redundancy in terms of the signal level but ignore the consideration of visual perception. In this paper, we propose a novel digital human model-driven compression framework by using multi-granularity representations. A coarse-grained layer based on the Skinned Multi-Person Linear model (SMPL) is introduced to extract general structures, while the normal images are used to represent fine-grained details via pose consistency-based predictions. Then, the SMPL parameters and normal images are encoded to achieve high-quality compression at extremely low bitrates. Experimental results demonstrate that the proposed method provides better coding performance with superior perceptual quality compared to the state-of-the-art 3D model compression methods.