SPAN: siampillars attention network for 3D object tracking in point clouds

3D point clouds produce rich geometric information to address the scale variation in 2D image-based object tracking. Although siamese-based trackers are widely used and achieve great performance, their applications in 3D point clouds have not been seriously considered because of different data formats and structural information. To utilize a 2D siamese-based tracker for object tracking in raw point clouds, we propose a siampillars attention network in this paper. SPAN firstly converts 3D point clouds into 2D pseudo images so that 2D tracking methods can be applied. In response to the sparsity of raw point clouds, a separate attention module (SAM) consists of a height-and-width (HW) attention module, and a crosschannel attention module is designed to enrich the extracted features. A modulated deformable convolutional network (MDCN) is further applied to handle the deformations during tracking. The anchor-based region proposal network (RPN) with depth-wise correlation is deployed finally to locate the object and regress the 3D bounding box, which makes SPAN work single-shortly in an end-to-end learning manner. Our experiments on the KITTI dataset demonstrate the superiority of SPAN. SPAN runs with 46.6 frames per second (FPS) on a single NVIDIA 1080ti GPU. Codes are available at https://github.com/ZCHILLAXY/SPAN.