Occlusion Localization Based On Convolutional Neural Networks

In most convolutional neural networks (CNNs), the output is a single classification result by combining all the neuron activations in the last layer. As we know, local connectivity is an important characteristic of CNNs. Each neuron in the network corresponds to a local region in the original image. Hence, it is possible to simultaneously obtain local visibility of a target object by analyzing neuron activations in a vanilla network. In this paper, a method to localize partial occlusions based on an off-the shelf CNN is proposed. Unlike most existing foreground segmentation methods, it should be noted that both classification results and foreground estimation are simultaneously obtained with no deliberate foreground annotations and no extra network designs in this paper. The contributions of the paper are twofold: First, a method to obtain occlusion maps within regions of interest is developed based on a vanilla object classification network. Second, several strategies to infer occlusion maps based on the neuron activations are developed and tested. Preliminary results on both synthetic and GTSRB traffic signs show the potential of the developed methods to infer local occlusions based on an off-the-shelf CNN.