Embedded Self-Distillation in Compact Multibranch Ensemble Network for Remote Sensing Scene Classification

Remote sensing image classification task is challenging due to the characteristics of complex composition, so different geographic elements in the same image will interfere with each other, resulting in misclassification. To solve this problem, we propose a multibranch ensemble network to enhance the feature representation ability by fusing final output logits and intermediate feature maps. However, simply adding branches will increase the complexity of models and decline the inference efficiency. To reduce the complexity of multibranch network, we make multibranch share more weights and add feature augmentation modules to compensate for the lack of diversity caused by weight sharing. To improve the efficiency of inference, we embed self-distillation (SD) method to transfer knowledge from ensemble network to main branch. Through optimizing with SD, the main branch will have close performance as an ensemble network. In this way, we can cut other branches during inference. In addition, we simplify the process of SD and totally adopt two loss functions to self-distill the logits and feature maps. In this article, we design a compact multibranch ensemble network, which can be trained in an end-to-end manner. Then, we insert an SD method on output logits and feature maps. Our proposed architecture (ESD-MBENet) performs strongly on classification accuracy with compact design. Extensive experiments are applied on three benchmark remote sensing datasets, AID, NWPU-RESISC45, and UC-Merced with three classic baseline models, VGG16, ResNet50, and DenseNet121. Results prove that ESD-MBENet can achieve better accuracy than previous state-of-the-art complex deep learning models. Moreover, abundant visualization analyses make our method more convincing and interpretable.