A Survey of 3D Data Analysis and Understanding Based on Deep Learning

3D data analysis and understanding based on deep learning is a research hotspot in the field of digital geometry. Increasingly rich three-dimensional data, including single models and scene models, encourages us to use these abundant data to effectively process and analyze digital geometric models, such as 3D object classification, 3D object recognition, 3D shape retrieval, 3D shape segmentation, 3D shape matching and 3D modeling. How to obtain the information we need by analyzing large-scale 3D data is the key to solve tasks related to the fields of computer graphics and computer vision. With the successful application of deep learning in computer vision, it is imperative to extend it to the field of digital geometry processing. 3D data processing method based on deep learning is data-driven. It is no longer limited to a single three-dimensional model. Instead, a set of three-dimensional models is analyzed. Unlike image analysis and understanding based on deep learning, the key problem that needs to be solved of 3D data analysis and understanding based on deep learning is the diversity of data. Compared with regular two-dimensional images, the expression of three-dimensional data is diverse. The current related work is mostly based on the discrete representation of three-dimensional data. Different three-dimensional data representation methods and different digital geometry processing tasks have different requirements for deep learning networks. The method based on deep learning can extract feature mapping relationships and semantic correlations between these three-dimensional objects. The characteristics of the 3D model are learned, so that the attributes of the 3D model and the relationships between them can be effectively derived. There are some methods including extracting high-level features based on low-level features, structured representation of 3D data, dimensionality reduction for 3D data, fusion of multimodal features, and the method based on manifold and so on. In this paper, a comprehensive and in-depth review of 3D data processing based on deep learning is provided. We first summarize the commonly used 3D datasets and evaluation indicators of specific tasks, and investigate the 3D model feature descriptors. Then, starting from specific tasks, the existing three-dimensional data analysis and understanding network based on deep learning is reviewed according to different input data representation. Then we further summarize existing work from the perspective of 3D data representation. Meanwhile, the comparison, advantages and disadvantages of various methods are summarized. Finally, based on the research status at home and abroad, the problems existing in the existing research are expounded, and the future development trend is forecasted. © 2020, Science Press. All right reserved.