3D Orientation and Object Classification from Partial Model Point Cloud based on PointNet

In this paper, we propose a deep network based on PointNet to estimate the orientations and predict the object classes of 3D oriented objects using their partial model point clouds. More specific, our network exploits the advantages of PointNet to extract the global features of two kinds of point cloud: 1) 3D partial model orientation point cloud which is a part of a 3D object in an observed orientation and 2) full object model point cloud of the 3D object in the reference orientation which is referred to specify the orientations. We then associate the partial model point cloud global features with the corresponding reference global features by an association subnetwork, in which the association network takes the partial model global features as the input and output the corresponding reference feature reconstruction. We use this global feature reconstruction as aligned global features to infer the object classes of the partial model point cloud. To predict the orientation of an oriented point cloud from its partial model point cloud, we use the concatenation of partial model global features and the reference feature reconstruction as an optimal orientation features for network learning with orientation targets. Using the orientation dataset with partial model point clouds based on 3D ModelNet, our experiments have shown the better object classification performance comparing to the vanilla PointNet and the robustness of our proposed network in orientation estimation.