Multistage Fusion and Dissimilarity Regularization for Deep Learning

We propose a multistage fusion stream (MFS) and dissimilarity regularization (DisReg) for deep learning. The degree of similarity between the feature maps of a single-sensor stream is estimated using DisReg. DisReg is applied to the learning problems of each single-sensor stream, so they have distinct types of feature map. Each stage of the MFS fuses the feature maps extracted from single-sensor streams. The proposed scheme fuses information from heterogeneous sensors by learning new patterns that cannot be observed using only the feature map of a single-sensor stream. The proposed method is evaluated by testing its ability to automatically recognize targets in a synthetic aperture radar and infrared images. The superiority of the proposed fusion scheme is demonstrated by comparison with conventional algorithm.