3DCANN: A Spatio-Temporal Convolution Attention Neural Network for EEG Emotion Recognition

被引:96
|
作者
Liu, Shuaiqi [1 ,2 ]
Wang, Xu [3 ]
Zhao, Ling [3 ]
Li, Bing [2 ]
Hu, Weiming [2 ]
Yu, Jie [4 ,5 ]
Zhang, Yu-Dong [6 ]
机构
[1] Hebei Univ, Coll Elect & Informat Engn, Machine Vis Engn Res Ctr Hebei Prov, Baoding 071002, Peoples R China
[2] Chinese Acad Sci, Inst Automat, Natl Lab Pattern Recognit NLPR, Beijing 100190, Peoples R China
[3] Hebei Univ, Coll Elect & Informat Engn, Key Lab Digital Med Engn Hebei Prov, Baoding 071002, Peoples R China
[4] PLA Med Coll, Dept Intervent Ultrasound, Beijing 100853, Peoples R China
[5] Chinese Peoples Liberat Army Gen Hosp, Beijing 100853, Peoples R China
[6] Univ Leicester, Sch Informat, Leicester LE1 7RH, Leics, England
基金
中国国家自然科学基金;
关键词
Electroencephalography; Feature extraction; Emotion recognition; Convolution; Brain modeling; Deep learning; Neural networks; 3D convolution attention neural network; dual attention learning; EEG emotion recognition; spatio-temporal feature; FEATURE-EXTRACTION; CLASSIFICATION;
D O I
10.1109/JBHI.2021.3083525
中图分类号
TP [自动化技术、计算机技术];
学科分类号
0812 ;
摘要
Since electroencephalogram (EEG) signals can truly reflect human emotional state, emotion recognition based on EEG has turned into a critical branch in the field of artificial intelligence. Aiming at the disparity of EEG signals in various emotional states, we propose a new deep learning model named three-dimension convolution attention neural network (3DCANN) for EEG emotion recognition in this paper. The 3DCANN model is composed of spatio-temporal feature extraction module and EEG channel attention weight learning module, which can extract the dynamic relation well among multi-channel EEG signals and the internal spatial relation of multi-channel EEG signals during continuous period time. In this model, the spatio-temporal features are fused with the weights of dual attention learning, and the fused features are input into the softmax classifier for emotion classification. In addition, we utilize SJTU Emotion EEG Dataset (SEED) to appraise the feasibility and effectiveness of the proposed algorithm. Finally, experimental results display that the 3DCANN method has superior performance over the state-of-the-art models in EEG emotion recognition.
引用
收藏
页码:5321 / 5331
页数:11
相关论文
共 50 条
  • [21] RAMST-CNN: A Residual and Multiscale Spatio-Temporal Convolution Neural Network for Personal Identification with EEG
    Zhu, Yuxuan
    Peng, Yong
    Song, Yang
    Ozawa, Kenji
    Kong, Wanzeng
    IEICE TRANSACTIONS ON FUNDAMENTALS OF ELECTRONICS COMMUNICATIONS AND COMPUTER SCIENCES, 2021, E104A (02) : 563 - 571
  • [22] Attention guided spatio-temporal network for 3D signature recognition
    Singh, Aradhana Kumari
    Koundal, Deepika
    MULTIMEDIA TOOLS AND APPLICATIONS, 2024, 83 (11) : 33985 - 33997
  • [23] Attention guided spatio-temporal network for 3D signature recognition
    Aradhana Kumari Singh
    Deepika Koundal
    Multimedia Tools and Applications, 2024, 83 : 33985 - 33997
  • [24] Neural network classification of spatio-temporal EEG readiness potentials
    Barreto, AB
    Taberner, AM
    Vicente, LM
    PROCEEDINGS OF THE 1996 FIFTEENTH SOUTHERN BIOMEDICAL ENGINEERING CONFERENCE, 1996, : 73 - 76
  • [25] EEG Source Localization Using Spatio-Temporal Neural Network
    Song Cui
    Lijuan Duan
    Bei Gong
    Yuanhua Qiao
    Fan Xu
    Juncheng Chen
    Changming Wang
    中国通信, 2019, 16 (07) : 131 - 143
  • [26] EEG Source Localization Using Spatio-Temporal Neural Network
    Cui, Song
    Duan, Lijuan
    Gong, Bei
    Qiao, Yuanhua
    Xu, Fan
    Chen, Juncheng
    Wang, Changming
    CHINA COMMUNICATIONS, 2019, 16 (07) : 131 - 143
  • [27] Spatio-Temporal Sparse Graph Convolution Network for Hand Gesture Recognition
    Ikne, Omar
    Slama, Rim
    Saoudi, Hichem
    Wannous, Hazem
    2024 IEEE 18TH INTERNATIONAL CONFERENCE ON AUTOMATIC FACE AND GESTURE RECOGNITION, FG 2024, 2024,
  • [28] Potential emotion recognition based on the fusion of the spatio-temporal neural network and facial pulse signals
    Song J.
    Wang F.
    Niu J.
    Shi Z.
    Ma J.
    Xi'an Dianzi Keji Daxue Xuebao/Journal of Xidian University, 2021, 48 (04): : 159 - 167
  • [29] Spatio-Temporal Image-Based Encoded Atlases for EEG Emotion Recognition
    Avola, Danilo
    Cinque, Luigi
    Mambro, Angelo Di
    Fagioli, Alessio
    Marini, Marco Raoul
    Pannone, Daniele
    Fanini, Bruno
    Foresti, Gian Luca
    INTERNATIONAL JOURNAL OF NEURAL SYSTEMS, 2024, 34 (05)
  • [30] A novel spatio-temporal convolutional neural framework for multimodal emotion recognition
    Sharafi, Masoumeh
    Yazdchi, Mohammadreza
    Rasti, Reza
    Nasimi, Fahimeh
    BIOMEDICAL SIGNAL PROCESSING AND CONTROL, 2022, 78