Multi-Scale Segmentation of Episodic Video Instance through Polarized Self-Attention Manipulation

Video instance segmentation (VIS) is a key technology for developing vision systems of intelligent robots, and ones deployed with video instance segmentation algorithms can accurately perform highly complex robotic tasks, such as target tracking and obstacle avoidance. The imaging results that are acquired during environment perception when the robot moves autonomously in a specific scene is easily affected by its own motion speed, shooting angle, distance from the target position and the target relative motion speed, resulting in randomness problems such as topological deformation and scale scaling of the captured moving targets in general. For the same target instance exsting across the adjacent static frames in a series of video frames, it is generally diverse and uncertain in terms of the discernible feature representations which are learned by the model of common methods. Existing video instance segmentation models mostly emphasize more on temporal interaction methods such as inter-frame mask propagation or multi-scale feature tracking, through which the deep semantic parsing of topological instances and contour discrimination of targets in multi-scale are neglected, thus the effective attention to high-level fine-grained features and accurate localization of low-level spatial information are seriously limited. To address the above mentioned issues, a model based on polarized self-attention manipulation for multi-scale video instance segmentation (named PSAM-Net) is proposed in this paper. First of all, in order to establish the positional correlation information between arbitrary non-linear spaces and dependences among orthogonal channels, we propose a single-stage and a cascaded polarized self-attention manipulation mechanisms, which are respectively embedded in the residual network after each residual block in an optimal form. The above mentioned measures benefit to overcome the dispersion of regression distribution for fine-grained features in the feature maps of deep levels and enhance the feature focusing ability of the model on key regions, so as to complete the task of deep semantic parsing of topological instances. Secondly, a multi-scale spatial location model of multi-granularity spatial information is established, which can make up for the lack of low-level feature space location and indistinct instance edge information caused by the feature flow from the top to the bottom of the feature pyramid networks. Through this can we achieve better requirements of target location detection and contour segmentation for foreground objects under different scales. Finally, we construct the episodic video dataset of animal instances extracted from Youtube-VIS, and extensive experiments are conducted to verify the competitive performance of our method. Compared with the YolactEdge benchmark model, the comprehensive testing results obtained by the PSAM-Net model show multiple improvements, where the average detection reaches 44.06% which is increased by 6.08%, and the average segmentation accuracy is increased by 8.87% to reach 44.41% respectively. In addition, the proposed PSAM-Net is capable of processing 80 video frames per second, which is well above the real-time requirements for video instance segmentation task. The model in this paper realizes real-time high-precision segmentation of video instances, and provides an effective theoretical basis and certain reference value for the autonomous environment perception of intelligent mobile robots. © 2022, Science Press. All right reserved.