Benchmarking of objective quality metrics for point cloud compression

Point cloud is a promising imaging modality for the representation of 3D media. The vast volume of data associated with it requires efficient compression solutions, with lossy algorithms leading to larger bit-rate savings at the expense of visual impairments. While conventional encoding approaches rely on efficient data structures, recent methods have incorporated deep learning for rate-distortion optimization, while inducing perceptual degradations of different natures. To measure the magnitude of such distortions, subjective or objective quality evaluation methodologies are employed. Lately, a remarkable amount of efforts has been devoted to the development of point cloud objective quality metrics, which have been reported to attain high prediction accuracy. However, their performance and generalization capabilities haven't been evaluated yet in presence of artifacts from learning-based codecs. In this study, we tackle this matter by conducting the first crowdsourcing experiment for point cloud quality reported in the literature, in order to obtain subjective ratings for point cloud models whose topology and color attributes are encoded by both conventional and data-driven methods. Using the subjective scores as ground truth, the performance of a large pool of state-of-the-art quality metrics is rigorously benchmarked, drawing useful insights regarding their efficacy.