Multi-view brain functional connectivity and hierarchical fusion for EEG-based emotion recognition

被引:0
|
作者
Fu, Baole [1 ,4 ]
Yu, Xiangkun [2 ,4 ]
Wu, Feng [1 ,4 ]
Liu, Yinhua [1 ,2 ,3 ,4 ]
机构
[1] Qingdao Univ, Sch Automat, Qingdao 266071, Peoples R China
[2] Qingdao Univ, Coll Comp Sci & Technol, Qingdao 266071, Peoples R China
[3] Shandong Key Lab Ind Control Technol, Qingdao 266071, Peoples R China
[4] Qingdao Univ, Inst Future, Qingdao 266071, Peoples R China
关键词
Emotion recognition; Multi-view; Brain functional connectivity; Cross-attention; Feature fusion; Electroencephalogram;
D O I
10.1016/j.measurement.2024.116046
中图分类号
T [工业技术];
学科分类号
08 ;
摘要
Emotional states evolve gradually from their onset to full manifestation, reflected in changes in brain functional connectivity. Existing research often focuses only on the final emotional state, missing the rich, dynamic information of emotional evolution. This paper proposes a novel network framework leveraging global and local brain functional connectivity through a multi-view approach. We introduce a global and local connectivity modeling method using electroencephalogram signals to simulate brain connectivity changes across time and specific moments. A residual-based local detail feature extractor combined with a coordinate attention module captures long-range dependencies and maintains positional accuracy. A global feature extractor is also designed for the effective extraction of emotional information. Finally, a hierarchical multi-scale cross-attention feature fusion module integrates features across different levels to enhance emotion recognition performance and robustness. Extensive experiments on three publicly available datasets demonstrate the framework's superiority and generalization capability.
引用
收藏
页数:13
相关论文
共 50 条
  • [21] HIERARCHICAL AND MULTI-VIEW DEPENDENCY MODELLING NETWORK FOR CONVERSATIONAL EMOTION RECOGNITION
    Ruan, Yu-Ping
    Zheng, Shu-Kai
    Li, Taihao
    Wang, Fen
    Pei, Guanxiong
    2022 IEEE INTERNATIONAL CONFERENCE ON ACOUSTICS, SPEECH AND SIGNAL PROCESSING (ICASSP), 2022, : 7032 - 7036
  • [22] CDBA: a novel multi-branch feature fusion model for EEG-based emotion recognition
    Huang, Zhentao
    Ma, Yahong
    Su, Jianyun
    Shi, Hangyu
    Jia, Shanshan
    Yuan, Baoxi
    Li, Weisu
    Geng, Jingzhi
    Yang, Tingting
    FRONTIERS IN PHYSIOLOGY, 2023, 14
  • [23] PNN for EEG-based Emotion Recognition
    Zhang, Jianhai
    Chen, Ming
    Hu, Sanqing
    Cao, Yu
    2016 IEEE INTERNATIONAL CONFERENCE ON SYSTEMS, MAN, AND CYBERNETICS (SMC), 2016, : 2319 - 2323
  • [24] EEG-based Emotion Word Recognition
    Dong, Weiwei
    Wang, Panpan
    Zhang, Yazhou
    Wang, Tianshu
    Niu, Jiabin
    Zhang, Shengnan
    PROCEEDINGS OF THE 2018 INTERNATIONAL CONFERENCE ON ADVANCED CONTROL, AUTOMATION AND ARTIFICIAL INTELLIGENCE (ACAAI 2018), 2018, 155 : 121 - 124
  • [25] EMOTION RECOGNITION BASED ON MULTI-VIEW BODY GESTURES
    Shen, Zhijuan
    Cheng, Jun
    Hu, Xiping
    Dong, Qian
    2019 IEEE INTERNATIONAL CONFERENCE ON IMAGE PROCESSING (ICIP), 2019, : 3317 - 3321
  • [26] EEG-Based Emotion Recognition With Haptic Vibration by a Feature Fusion Method
    Li, Dahua
    Yang, Zhiyi
    Hou, Fazheng
    Kang, Qiaoju
    Liu, Shuang
    Song, Yu
    Gao, Qiang
    Dong, Enzeng
    IEEE TRANSACTIONS ON INSTRUMENTATION AND MEASUREMENT, 2022, 71
  • [27] EEG-based Brain Functional Connectivity during Distracted Driving
    Almahasneh, Hossam
    Kamel, Nidal
    Walter, Nicolas
    Malik, Aamir Saeed
    2015 IEEE INTERNATIONAL CONFERENCE ON SIGNAL AND IMAGE PROCESSING APPLICATIONS (ICSIPA), 2015, : 274 - 277
  • [28] Emotion Recognition and Dynamic Functional Connectivity Analysis Based on EEG
    Liu, Xucheng
    Li, Ting
    Tang, Cong
    Xu, Tao
    Chen, Peng
    Bezerianos, Anastasios
    Wang, Hongtao
    IEEE ACCESS, 2019, 7 : 143293 - 143302
  • [29] Hierarchical Dynamic Graph Convolutional Network With Interpretability for EEG-Based Emotion Recognition
    Ye, Mengqing
    Chen, C. L. Philip
    Zhang, Tong
    IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS, 2022, : 1 - 12
  • [30] Transformers for EEG-Based Emotion Recognition: A Hierarchical Spatial Information Learning Model
    Wang, Zhe
    Wang, Yongxiong
    Hu, Chuanfei
    Yin, Zhong
    Song, Yu
    IEEE SENSORS JOURNAL, 2022, 22 (05) : 4359 - 4368