Multi-level adversarial attention cross-modal hashing

Deep cross-modal hashing has made great progress in recent years due to the development of deep learning and efficient hashing algorithms. However, most of the existing methods only focus on the feature distribution between modalities, and ignore the fine grain information in each modality. To solve this problem, we propose a multi-level adversarial attention cross-modal hashing (MAAH). First, we design a modality-attention module to find the fine-grained information of each modality. Specifically, we use the channel attention mechanism to divide modality information into relevant and irrelevant representation, in which the irrelevant representation is the fine-grained information of the modality. Then, we design a modality-adversary module to supplement the fine-grained information of each modality. In this module, intra-modal adversarial learning can supplement the relevant representation of modalities, and inter-modal adversarial learning can make the distribution of the relevant representation of each modality more uniform. Experimental results on three widely used datasets demonstrate the superiority of the proposed method.