Invisible Black-box Backdoor Attack against Deep Cross-modal Hashing Retrieval

被引:0
|
作者
Wang, Tianshi [1 ]
Li, Fengling [2 ]
Zhu, Lei [1 ]
Li, Jingjing [3 ]
Zhang, Zheng [4 ]
Shen, Heng Tao [3 ]
机构
[1] Shandong Normal Univ, Jinan 250358, Peoples R China
[2] Univ Technol Sydney, Sydney, NSW 2007, Australia
[3] Univ Elect Sci & Technol China, Chengdu 611731, Peoples R China
[4] Harbin Inst Technol Shenzhen, Shenzhen 518055, Peoples R China
基金
中国国家自然科学基金;
关键词
Backdoor attack; black-box attack; imperceptible trigger; deep crossmodal; hashing retrieval;
D O I
10.1145/3650205
中图分类号
TP [自动化技术、计算机技术];
学科分类号
0812 ;
摘要
Deep cross-modal hashing has promoted the field of multi-modal retrieval due to its excellent efficiency and storage, but its vulnerability to backdoor attacks is rarely studied. Notably, current deep cross-modal hashing methods inevitably require large-scale training data, resulting in poisoned samples with imperceptible triggers that can easily be camouflaged into the training data to bury backdoors in the victim model. Nevertheless, existing backdoor attacks focus on the uni-modal vision domain, while the multi-modal gap and hash quantization weaken their attack performance. In addressing the aforementioned challenges, we undertake an invisible black-box backdoor attack against deep cross-modal hashing retrieval in this article. To the best of our knowledge, this is the first attempt in this research field. Specifically, we develop a flexible trigger generator to generate the attacker's specified triggers, which learns the sample semantics of the nonpoisoned modality to bridge the cross-modal attack gap. Then, we devise an input-aware injection network, which embeds the generated triggers into benign samples in the form of sample-specific stealth and realizes cross-modal semantic interaction between triggers and poisoned samples. Owing to the knowledge-agnostic of victim models, we enable any cross-modal hashing knockoff to facilitate the black-box backdoor attack and alleviate the attackweakening of hash quantization. Moreover, we propose a confusing perturbation andmask strategy to induce the high-performance victim models to focus on imperceptible triggers in poisoned samples. Extensive experiments on benchmark datasets demonstrate that ourmethod has a state-of-the-art attack performance against deep cross-modal hashing retrieval. Besides, we investigate the influences of transferable attacks, few-shot poisoning, multi-modal poisoning, perceptibility, and potential defenses on backdoor attacks. Our codes and datasets are available at https://github.com/tswang0116/IB3A
引用
收藏
页数:27
相关论文
共 50 条
  • [11] A triple fusion model for cross-modal deep hashing retrieval
    Hufei Wang
    Kaiqiang Zhao
    Dexin Zhao
    Multimedia Systems, 2023, 29 : 347 - 359
  • [12] Cross-Modal Hashing Retrieval Based on Deep Residual Network
    Li, Zhiyi
    Xu, Xiaomian
    Zhang, Du
    Zhang, Peng
    COMPUTER SYSTEMS SCIENCE AND ENGINEERING, 2021, 36 (02): : 383 - 405
  • [13] Deep semantic hashing with dual attention for cross-modal retrieval
    Jiagao Wu
    Weiwei Weng
    Junxia Fu
    Linfeng Liu
    Bin Hu
    Neural Computing and Applications, 2022, 34 : 5397 - 5416
  • [14] Deep semantic similarity adversarial hashing for cross-modal retrieval
    Qiang, Haopeng
    Wan, Yuan
    Xiang, Lun
    Meng, Xiaojing
    NEUROCOMPUTING, 2020, 400 : 24 - 33
  • [15] Discriminative deep asymmetric supervised hashing for cross-modal retrieval
    Qiang, Haopeng
    Wan, Yuan
    Liu, Ziyi
    Xiang, Lun
    Meng, Xiaojing
    Knowledge-Based Systems, 2022, 204
  • [16] Unsupervised Deep Imputed Hashing for Partial Cross-modal Retrieval
    Chen, Dong
    Cheng, Miaomiao
    Min, Chen
    Jing, Liping
    2020 INTERNATIONAL JOINT CONFERENCE ON NEURAL NETWORKS (IJCNN), 2020,
  • [17] A triple fusion model for cross-modal deep hashing retrieval
    Wang, Hufei
    Zhao, Kaiqiang
    Zhao, Dexin
    MULTIMEDIA SYSTEMS, 2023, 29 (01) : 347 - 359
  • [18] A novel deep translated attention hashing for cross-modal retrieval
    Haibo Yu
    Ran Ma
    Min Su
    Ping An
    Kai Li
    Multimedia Tools and Applications, 2022, 81 : 26443 - 26461
  • [19] Discriminative deep asymmetric supervised hashing for cross-modal retrieval
    Qiang, Haopeng
    Wan, Yuan
    Liu, Ziyi
    Xiang, Lun
    Meng, Xiaojing
    KNOWLEDGE-BASED SYSTEMS, 2020, 204
  • [20] Deep semantic hashing with dual attention for cross-modal retrieval
    Wu, Jiagao
    Weng, Weiwei
    Fu, Junxia
    Liu, Linfeng
    Hu, Bin
    NEURAL COMPUTING & APPLICATIONS, 2022, 34 (07): : 5397 - 5416