Improved image classification with 4D light-field and interleaved convolutional neural network

Image classification is a well-studied problem. However, there remains challenges for some special categories of images. This paper proposes a new deep convolutional neural network to improve image classification using extra light-field angular information. The proposed network model employs transfer learning by replacing the fully connected layer of a VGG network with a set of interleaved spatial-angular filters. The resulting model takes advantage of both the spatial and angular information of light-field images (LFIs), thus providing more accurate classification performance over traditional models. To evaluate the proposed network model, we established a light-field image dataset, currently consisting of 560 captured LFIs, which have been divided into 11 labeled categories. Based on this dataset, our experimental results show that the proposed LFI model yields an average of 92% classification accuracy as oppose to 84% from the model using traditional 2D images and 85% from the model using stereo pair images. In particular, on classifying challenging objects such as the “screen” images, the proposed LFI model demonstrated to have significant improvement of 16% and 12% respectively over the 2D image model and the stereo image model.