Deepfake source detection in a heart beat

被引:0
|
作者
Ciftci, Umur Aybars [1 ]
Demir, Ilke [2 ]
Yin, Lijun [1 ]
机构
[1] Binghamton Univ, Comp Sci Dept, Binghamton, NY USA
[2] Intel Corp, Intel Labs, Santa Clara, CA 95054 USA
来源
VISUAL COMPUTER | 2024年 / 40卷 / 04期
基金
美国国家科学基金会;
关键词
Deepfakes; Detection; Source detection; Generative models; Biological detector; PPG; GAN fingerprint; NETWORKS;
D O I
10.1007/s00371-023-02981-0
中图分类号
TP31 [计算机软件];
学科分类号
081202 ; 0835 ;
摘要
Fake portrait video generation techniques are posing a new threat to society as photorealistic deepfakes are being used for political propaganda, celebrity imitation, forged pieces of evidences, and other identity-related manipulations. Despite these generation techniques, some detection approaches have also been proven useful due to their high classification accuracy. Nevertheless, almost no effort has been spent tracking down the source of deepfakes. We propose an approach not only to separate deepfakes from real videos, but also to discover the specific generative model behind deepfakes. Some pure deep learning-based approaches try to classify deepfakes using CNNs which actually learn the residuals of the generator. Our key observation is that the spatiotemporal patterns in biological signals can be conceived as a representative projection of the residuals. To justify this observation, we extract PPG cells from real and fake videos and feed these to a state-of-the-art classification network, with an attempt to detect which generative model was used to create a certain fake video. Our results indicate that our approach can detect fake videos with 97.29% accuracy and the source model with 93.39% accuracy. We further evaluate and compare our approach on six datasets to assess its expansibility with new models and generalizability across skin tones and genders, run ablation studies for various components, and analyze its robustness toward compression, landmark noise, and postprocessing operations. The experiments show the superior performance of our proposed approach as compared to the state of the art.
引用
收藏
页码:2733 / 2750
页数:18
相关论文
共 50 条
  • [41] Probabilistic model-based approach for heart beat detection
    Chen, Hugh
    Erol, Yusuf
    Shen, Eric
    Russell, Stuart
    PHYSIOLOGICAL MEASUREMENT, 2016, 37 (09) : 1404 - 1421
  • [42] Hepatitis C and the heart: to beat or not to beat
    Kcomt, W
    Nahavandi, AA
    Myaing, M
    Alkhalil, C
    Stein, D
    INTERNATIONAL JOURNAL OF CARDIOLOGY, 2004, 96 (02) : 147 - 149
  • [43] DeepFake-Adapter: Dual-Level Adapter for DeepFake Detection
    Shao, Rui
    Wu, Tianxing
    Nie, Liqiang
    Liu, Ziwei
    INTERNATIONAL JOURNAL OF COMPUTER VISION, 2025,
  • [44] Multimodal heart beat detection using signal quality indices
    Johnson, Alistair E. W.
    Behar, Joachim
    Andreotti, Fernando
    Clifford, Gari D.
    Oster, Julien
    PHYSIOLOGICAL MEASUREMENT, 2015, 36 (08) : 1665 - 1677
  • [45] Explainable AI for DeepFake Detection
    Mansoor, Nazneen
    Iliev, Alexander I.
    APPLIED SCIENCES-BASEL, 2025, 15 (02):
  • [46] The Face Deepfake Detection Challenge
    Guarnera, Luca
    Giudice, Oliver
    Guarnera, Francesco
    Ortis, Alessandro
    Puglisi, Giovanni
    Paratore, Antonino
    Bui, Linh M. Q.
    Fontani, Marco
    Coccomini, Davide Alessandro
    Caldelli, Roberto
    Falchi, Fabrizio
    Gennaro, Claudio
    Messina, Nicola
    Amato, Giuseppe
    Perelli, Gianpaolo
    Concas, Sara
    Cuccu, Carlo
    Orru, Giulia
    Marcialis, Gian Luca
    Battiato, Sebastiano
    JOURNAL OF IMAGING, 2022, 8 (10)
  • [47] Aggregating Layers for Deepfake Detection
    Jevnisek, Amir
    Avidan, Shai
    2022 26TH INTERNATIONAL CONFERENCE ON PATTERN RECOGNITION (ICPR), 2022, : 2027 - 2033
  • [48] Beat-to-beat heart rate detection in multi-lead abdominal fetal ECG recordings
    Peters, C. H. L.
    van Laar, J. O. E. H.
    Vullings, R.
    Oei, S. G.
    Wijn, P. F. F.
    MEDICAL ENGINEERING & PHYSICS, 2012, 34 (03) : 333 - 338
  • [49] A Personalized Beat-to-Beat Heart Rate Detection System From Ballistocardiogram for Smart Home Applications
    Xie, Qingsong
    Wang, Min
    Zhao, Yang
    He, Ziran
    Li, Yongfu
    Wang, Guoxing
    Lian, Yong
    IEEE TRANSACTIONS ON BIOMEDICAL CIRCUITS AND SYSTEMS, 2019, 13 (06) : 1593 - 1602
  • [50] An online PPGI approach for camera based heart rate monitoring using beat-to-beat detection
    Bloecher, Timon
    Schneider, Johannes
    Schinle, Markus
    Stork, Wilhelm
    2017 IEEE SENSORS APPLICATIONS SYMPOSIUM (SAS), 2017,