The Syncretic Effect of Dual-Source Data on Affective Computing in Online Learning Contexts: A Perspective From Convolutional Neural Network With Attention Mechanism

被引:2
|
作者
Zhai, Xuesong [1 ,2 ,3 ]
Xu, Jiaqi [1 ]
Chen, Nian-Shing [4 ]
Shen, Jun [5 ]
Li, Yan [1 ]
Wang, Yonggu [6 ]
Chu, Xiaoyan [1 ]
Zhu, Yumeng [1 ]
机构
[1] Zhejiang Univ, Coll Educ, Hangzhou, Zhejiang, Peoples R China
[2] Anhui Xinhua Univ, Hefei, Anhui, Peoples R China
[3] Anhui Jianzhu Univ, Hefei, Anhui, Peoples R China
[4] Natl Taiwan Normal Univ, Program Learning Sci, Yunlin, Taiwan
[5] Univ Wollongong, Sch Comp & Informat Technol, Wollongong, NSW, Australia
[6] Zhejiang Univ Technol, Coll Educ Sci & Technol, Hangzhou, Zhejiang, Peoples R China
基金
中国国家自然科学基金;
关键词
affective computing; multimodal; dual-source data; fusion method; attention mechanism; EMOTION; FUSION; FRAMEWORK; HEAD;
D O I
10.1177/07356331221115663
中图分类号
G40 [教育学];
学科分类号
040101 ; 120403 ;
摘要
Affective computing (AC) has been regarded as a relevant approach to identifying online learners' mental states and predicting their learning performance. Previous research mainly used one single-source data set, typically learners' facial expression, to compute learners' affection. However, a single facial expression may represent different affections in various head poses. This study proposed a dual-source data approach to solve the problem. Facial expression and head pose are two typical data sources that can be captured from online learning videos. The current study collected a dual-source data set of facial expressions and head poses from an online learning class in a middle school. A deep learning neural network using AlexNet with an attention mechanism was developed to verify the syncretic effect on affective computing of the proposed dual-source fusion strategy. The results show that the dual-source fusion approach significantly outperforms the single-source approach based on the AC recognition accuracy between the two approaches (dual-source approach using Attention-AlexNet model 80.96%; single-source approach, facial expression 76.65% and head pose 64.34%). This study contributes to the theoretical construction of the dual-source data fusion approach, and the empirical validation of the effect of the Attention-AlexNet neural network approach on affective computing in online learning contexts.
引用
收藏
页码:466 / 493
页数:28
相关论文
共 5 条
  • [1] Prediction of the Improvement Effect of Visualization Technology on Online Learning from the Perspective of the Neural Network
    Yang, Fan
    Zhao, Wei
    [J]. MATHEMATICAL PROBLEMS IN ENGINEERING, 2022, 2022
  • [2] Prediction of the Improvement Effect of Visualization Technology on Online Learning from the Perspective of the Neural Network
    Yang, Fan
    Zhao, Wei
    [J]. MATHEMATICAL PROBLEMS IN ENGINEERING, 2022, 2022
  • [3] Optimization of convolutional neural network with dual attention mechanism: Estimation of chlorophyll-a concentration in the Taiwan Strait using MODIS data
    Yu, Dingfeng
    Jiang, Guanghao
    Gao, Hao
    Ren, Lirong
    Chen, Chen
    Yang, Lei
    Zhou, Maosheng
    Pan, Shunqi
    [J]. ESTUARINE COASTAL AND SHELF SCIENCE, 2024, 300
  • [4] An Early Warning System for Earthquake Prediction from Seismic Data Using Batch Normalized Graph Convolutional Neural Network with Attention Mechanism (BNGCNNATT)
    Bilal, Muhammad Atif
    Ji, Yanju
    Wang, Yongzhi
    Akhter, Muhammad Pervez
    Yaqub, Muhammad
    [J]. SENSORS, 2022, 22 (17)
  • [5] A Hybrid-Attention-LSTM-Based Deep Convolutional Neural Network to Extract Modal Frequencies from Limited Data Using Transfer Learning
    Dizaji, Mehrdad Shafiei
    Mao, Zhu
    [J]. ROTATING MACHINERY, OPTICAL METHODS & SCANNING LDV METHODS, VOL 6, 2023, : 167 - 173