Facial expression recognition based on a multi-task global-local network

被引:49
|
作者
Yu, Mingjing [1 ,2 ]
Zheng, Huicheng [1 ,2 ]
Peng, Zhifeng [1 ,2 ]
Dong, Jiayu [1 ,2 ]
Du, Heran [1 ,2 ]
机构
[1] Sun Yat Sen Univ, Sch Data & Comp Sci, Guangzhou, Peoples R China
[2] Minist Educ, Key Lab Machine Intelligence & Adv Comp, Beijing, Peoples R China
基金
中国国家自然科学基金;
关键词
Facial expression recognition; Global-local network; Spatial-temporal representation; MODEL;
D O I
10.1016/j.patrec.2020.01.016
中图分类号
TP18 [人工智能理论];
学科分类号
081104 ; 0812 ; 0835 ; 1405 ;
摘要
Facial expression recognition plays an important role in intelligent human-computer interaction. The clues for understanding facial expressions lie not in global facial appearance, but also in local informative dynamics among different but confusing expressions. In this paper, we design a multi-task learning framework for global-local representation of facial expressions. First, a shared shallow module is designed to learn information from local regions and the global image. Then we construct a part-based module, which processes critical local regions including the eyes, the nose, and the mouth to extract local informative dynamics related to facial expressions. A global face module is proposed to extract global appearance features related to expressions. The proposed network extracts both local-global and spatio-temporal information for a discriminative and robust representation of facial expressions. Through properly fusing these modules into a system, we have achieved competitive results on the CK+ and Oulu-CASIA databases. (c) 2020 Elsevier B.V. All rights reserved.
引用
收藏
页码:166 / 171
页数:6
相关论文
共 50 条
  • [1] Co-attentive multi-task convolutional neural network for facial expression recognition
    Yu, Wenmeng
    Xu, Hua
    [J]. PATTERN RECOGNITION, 2022, 123
  • [2] CFNet: Facial expression recognition via constraint fusion under multi-task network
    Xiao, Junhao
    Gan, Chenquan
    Zhu, Qingyi
    Zhu, Ye
    Liu, Gang
    [J]. APPLIED SOFT COMPUTING, 2023, 141
  • [3] Facial Expression Recognition Based on Multi-task Learning with Spatial Feature Disentanglement
    Chen Huawen
    Ji Ruirui
    Wang Zhilian
    [J]. TWELFTH INTERNATIONAL CONFERENCE ON GRAPHICS AND IMAGE PROCESSING (ICGIP 2020), 2021, 11720
  • [4] Multi-Task and Attention Collaborative Network for Facial Emotion Recognition
    Wang, Xiaohua
    Yu, Cong
    Gu, Yu
    Hu, Min
    Ren, Fuji
    [J]. IEEJ TRANSACTIONS ON ELECTRICAL AND ELECTRONIC ENGINEERING, 2021, 16 (04) : 568 - 576
  • [5] Facial Expression Recognition by Regional Attention and Multi-task Learning
    Cui, Longlei
    Tian, Ying
    [J]. ENGINEERING LETTERS, 2021, 29 (03) : 919 - 925
  • [6] Discriminative deep multi-task learning for facial expression recognition
    Zheng, Hao
    Wang, Ruili
    Ji, Wanting
    Zong, Ming
    Wong, Wai Keung
    Lai, Zhihui
    Lv, Hexin
    [J]. INFORMATION SCIENCES, 2020, 533 : 60 - 71
  • [7] Robust facial expression recognition with global-local joint representation learning
    Chunxiao Fan
    Zhenxing Wang
    Jia Li
    Shanshan Wang
    Xiao Sun
    [J]. Multimedia Systems, 2023, 29 : 3069 - 3079
  • [8] Robust facial expression recognition with global-local joint representation learning
    Fan, Chunxiao
    Wang, Zhenxing
    Li, Jia
    Wang, Shanshan
    Sun, Xiao
    [J]. MULTIMEDIA SYSTEMS, 2023, 29 (05) : 3069 - 3079
  • [9] MTMSN: Multi-Task and Multi-Modal Sequence Network for Facial Action Unit and Expression Recognition
    Jin, Yue
    Zheng, Tianqing
    Gao, Chao
    Xu, Guoqiang
    [J]. 2021 IEEE/CVF INTERNATIONAL CONFERENCE ON COMPUTER VISION WORKSHOPS (ICCVW 2021), 2021, : 3590 - 3595
  • [10] HIERARCHICAL MULTI-TASK NETWORK FOR RACE, GENDER AND FACIAL ATTRACTIVENESS RECOGNITION
    Xu, Lu
    Fan, Heng
    Xiang, Jinhai
    [J]. 2019 IEEE INTERNATIONAL CONFERENCE ON IMAGE PROCESSING (ICIP), 2019, : 3861 - 3865