Learning Strengths and Weaknesses of Classifiers for RGB-D Semantic Segmentation

3D scene understanding is an open challenge in the field of computer vision. Most of the focus is on 2D methods in which the semantic labeling of each RGB pixel is considered. But, in this paper, the 3D semantic labeling of RGB-D images is considered. In the proposed method, to extract some meaningful features, the superpixel generation algorithm is applied to the RGB image to segment it into a set of disjoint pixels. After that, the set of three powerful classifiers are utilized to semantically label each superpixel. In the proposed method, the probability outputs of these classifiers are concatenated as the novel feature vector for each superpixel. Consequently, to analyze the strengths and weaknesses of each classifier, the conditional random field framework is used to improve the contextual relationships among neighboring superpixels. The unary potential function of the conditional random field is learned based on these new feature vectors. The proposed method is evaluated on the challenging NYU-V2 RGB-D dataset and improves the pixel average accuracy compared to previous methods.