Deep CRF-Graph Learning for Semantic Image Segmentation

We show that conditional random fields (CRFs) with learned heterogeneous graphs outperforms its pre-designated homogeneous counterparts with heuristics. Without introducing any additional annotations, we utilize four deep convolutional neural networks (CNNs) to learn the connections of one pixel to its left, top, upper-left, upper-right neighbors. The results are then fused to obtain the super-pixel-level CRF graphs. The model parameters of CRFs are learned via minimizing the negative pseudo-log-likelihood of the potential function. Our results show that the learned graph delivers significantly better segmentation results than CRFs with pre-designated graphs, and achieves state-of-the-art performance when combining with CNN features.