With the development of 3D technologies, 3D model recognition has attracted substantial attention in various areas, such as automatic driving, virtual/augmented reality, and computer-aided design. Many researchers are devoted to 3D model recognition and obtain some achievements in research. However, the abundant structure information of the 3D model also brings a huge challenge in model representation. In recent years, many researchers focus on classical computer vision technologies, which are utilized to represent the multi-view information of the 3D model. However, redundant visual information also brings a new challenge in model representation. In this paper, we focus on the multi-view 3D model data and propose a novel growing graph convolution network (GGCN) to handle the principal views selection problem, which can guarantee the performance of 3D model representation and effectively reduce the cost time. The proposed method mainly includes two modules: 1) principal views selection module: we utilize the selected views to describe the 3D model, which can effectively remove the redundant information and reduce computational complexity. 2) growing GCN module: we propose an effective growing GCN model, which focuses on gathering nodes that were less related to each other to ensure the result of multi-view fusion. It can indirectly retain the structure information and also reduce redundant information. In the process of graph growing, the GGCN model gradually adds view information to make up for the lack of characterization and guarantee the final performance. More specially, these two modules can guide each other to improve the performance of the principal views module and indirectly increase the final recognition accuracy. To evaluate the effectiveness of our proposed method, we test the classification accuracy and retrieval performance on the ModelNet40 dataset and ShapeNet dataset. The experimental results demonstrate the superiority of our proposed method.