Two-stream feature aggregation deep neural network for scene classification of remote sensing images

Scene classification of high-spatial resolution (HSR) images has a wide range of potential applications in various fields, and it has become a research hotspot in remote sensing community. Recently, deep transfer learning-based methods have attracted tremendous attention due to powerful ability of feature extraction. In this paper, a novel architecture termed two-stream feature aggregation deep neural network (TFADNN) is developed for HSR scene classification. The TFADNN method contains two parallel parts, including the stream of discriminative features and the stream of general features. In the first stream, the fully connected layers of pre-trained CNNs are replaced by a global average pooling layer to remove the limitation on the size of input images. As for the second stream, the multiscale nonlinear encoding based bag-of-visual-words (MNBoVW) model is proposed to process convolutional features, and the global representations can be obtained. Then, weighted fusion is adopted to integrate two-stream features. As a result, the TFADNN method can learn the discriminative features from HSR images with arbitrary sizes, and the experimental results on two challenging datasets indicate that the TFADNN method achieves satisfactory classification performance compared with some state-of-the-art methods. (C) 2020 Elsevier Inc. All rights reserved.