Learning spatial-frequency interaction for generalizable deepfake detection

被引:0
|
作者
Zhai, Tianbo [1 ]
Lu, Kaiyin [1 ]
Li, Jiajun [1 ]
Wang, Yukai [1 ]
Zhang, Wenjie [2 ]
Yu, Peipeng [1 ]
Xia, Zhihua [1 ]
机构
[1] Jinan Univ, Coll Cyber Secur, Engn Res Ctr Trustworthy AI, Minist Educ, Guangzhou 510632, Peoples R China
[2] Ningbo Univ, Coll Informat Sci & Engn, Ningbo, Peoples R China
基金
中国国家自然科学基金; 国家重点研发计划;
关键词
face recognition; image classification; image forensics; image processing; IMAGE FORGERY LOCALIZATION;
D O I
10.1049/ipr2.13276
中图分类号
TP18 [人工智能理论];
学科分类号
081104 ; 0812 ; 0835 ; 1405 ;
摘要
In recent years, face forgery detection has gained significant attention, resulting in considerable advancements. However, most existing methods rely on CNNs to extract artefacts from the spatial domain, overlooking the pervasive frequency-domain artefacts present in deepfake content, which poses challenges in achieving robust and generalized detection. To address these issues, we propose the dual-stream frequency-spatial fusion network is proposed for deepfake detection. The dual-stream frequency-spatial fusion network consists of three components: the spatial forgery feature extraction module, the frequency forgery feature extraction module, and the spatial-frequency feature fusion module. The spatial forgery feature extraction module employs spatial-channel attention to extract spatial domain features, targeting artefacts in the spatial domain. The frequency forgery feature extraction module leverages the focused linear attention to detect frequency domain anomalies in internal regions, enabling the identification of generated content. The spatial-frequency feature fusion module then fuses forgery features extracted from both the spatial and frequency domains, facilitating accurate detection of splicing artefacts and internally generated forgeries. This approach enhances the model's ability to more accurately capture forgery characteristics. Extensive experiments on several widely-used benchmarks demonstrate that our carefully designed network exhibits superior generalization and robustness, significantly improving deepfake detection performance.
引用
收藏
页码:4666 / 4679
页数:14
相关论文
共 50 条
  • [21] INTERACTION OF DIFFERENT SPATIAL-FREQUENCY SCALES IN MOTION PROCESSING
    KIM, JH
    WILSON, HR
    INVESTIGATIVE OPHTHALMOLOGY & VISUAL SCIENCE, 1992, 33 (04) : 1136 - 1136
  • [22] SI-Net: spatial interaction network for deepfake detection
    Wang, Jian
    Du, Xiaoyu
    Cheng, Yu
    Sun, Yunlian
    Tang, Jinhui
    MULTIMEDIA SYSTEMS, 2023, 29 (5) : 3139 - 3150
  • [23] SPATIAL-FREQUENCY DISCRIMINATION AND DETECTION - COMPARISON OF POSTADAPTATION THRESHOLDS
    REGAN, D
    BEVERLEY, KI
    JOURNAL OF THE OPTICAL SOCIETY OF AMERICA, 1983, 73 (12) : 1684 - 1690
  • [24] SPATIAL-FREQUENCY UNCERTAINTY EFFECTS IN THE DETECTION OF SINUSOIDAL GRATINGS
    DAVIS, ET
    GRAHAM, N
    VISION RESEARCH, 1981, 21 (05) : 705 - 712
  • [25] PERCEPTUAL-LEARNING SPECIFIC FOR ORIENTATION AND SPATIAL-FREQUENCY
    FIORENTINI, A
    BERARDI, N
    NATURE, 1980, 287 (5777) : 43 - 44
  • [26] Learning Spatial-Frequency Transformer for Visual Object Tracking
    Tang, Chuanming
    Wang, Xiao
    Bai, Yuanchao
    Wu, Zhe
    Zhang, Jianlin
    Huang, Yongmei
    IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY, 2023, 33 (09) : 5102 - 5116
  • [27] SI-Net: spatial interaction network for deepfake detection
    Jian Wang
    Xiaoyu Du
    Yu Cheng
    Yunlian Sun
    Jinhui Tang
    Multimedia Systems, 2023, 29 : 3139 - 3150
  • [28] EFFECTS OF ADAPTING TO SPATIAL-FREQUENCY GRATINGS ON FLICKER DETECTION
    CARRASCO, M
    KINCHLA, RA
    PERCEPTION, 1989, 18 (04) : 537 - 537
  • [29] Detection Performance of Spatial-Frequency Diversity MIMO Radar
    Liu, Hongwei
    Zhou, Shenghua
    Su, Hongtao
    Yu, Yao
    IEEE TRANSACTIONS ON AEROSPACE AND ELECTRONIC SYSTEMS, 2014, 50 (04) : 3137 - 3155
  • [30] TOWARDS GENERALIZABLE DEEPFAKE FACE FORGERY DETECTION WITH SEMI-SUPERVISED LEARNING AND KNOWLEDGE DISTILLATION
    Lin, Yuzhen
    Chen, Han
    Li, Bin
    Wu, Junqiang
    2022 IEEE INTERNATIONAL CONFERENCE ON IMAGE PROCESSING, ICIP, 2022, : 576 - 580