Preserving the Temporal Consistency of Video Sequences for Surgical Instruments Segmentation

Temporal consistency information plays an important role in video segmentation. Some previous automatic annotation methods inevitably caused the temporal consistency information loss of sequential frames when processing sparse labeled frames. Thus, as a supplement, interpolation method was proposed to compensate motion flows between two frames via interpolating high-precision frames and corresponding labels, but its quality is remained to be improved. We propose a new interpolation module to smooth the large motion between frames and preserve temporal consistency. First, we use an optical flow estimator to directly estimate the optical flow, and then refine the interpolation frame by combining the features of the original frames. We evaluate our method on the MICCAI EndoVis Robotic Instrument Segmentation Challenge dataset. Experimental results demonstrate our method can well preserve the temporal consistency information and outperform the state-of-the-art method in segmentation accuracy.