Semantic segmentation with Recurrent Neural Networks on RGB-D videos

Fully Convolutional Network (FCN) based on convolutional neural network is the mainstream method for image semantic segmentation, but its performance on RGB-D video is not good. In this paper, we propose a new semantic segmentation method for RGB-D video using a Recurrent Neural Network architecture. Our novel method combines visual simultaneous localization and mapping (SLAM) technology to transfer semantic information across frames, so the data association between each frame of RGB-D video can be captured. We also show that the proposed approach allows us to improve over existing method by a comparison study on a RGBD scene dataset.