A Coarse-to-Fine Framework for Point Voxel Transformer

To effectively solve the problem that the input point clouds in the traditional point voxel transformer model (PVT) appear to be quite redundant in spatial dimensions, which causes massive computation and memory costs, we propose a novel coarse-to-fine point voxel transformer framework(CF-PVT) to relieve computation and memory burden while retaining performance. Our CF-PVT implements network inference in a two-stage manner. In the coarse inference stage, the input point cloud is split into coarse-grained voxels for economic computation. If it cannot be identified well, important voxels containing rich information are identified by the Important Voxel Identification Module and further split into fine-grained voxels. We conduct extensive experiments on traditional classification and segmentation tasks. The experiments demonstrate that our CF-PVT framework is highly effective. For example, while maintaining similar accuracy, CF-PVT reduces 60.1% FLOPs, and 68.9% latency of PVT1 on the ModelNet40 dataset.