Attention Flow: End-to-End Joint Attention Estimation

被引：0
|
作者：
Sumer, Omer [1 ]
Gerjets, Peter [2 ]
Trautwein, Ulrich [1 ]
Kasneci, Enkelejda [1 ]
机构：
[1] Univ Tubingen, Tubingen, Germany
[2] Leibniz Inst Wissensmedien, Tubingen, Germany
来源：
2020 IEEE WINTER CONFERENCE ON APPLICATIONS OF COMPUTER VISION (WACV) | 2020年
关键词：
CHILDREN; MODEL;
D O I：
暂无
中图分类号：
TP18 [人工智能理论];
学科分类号：
081104 ; 0812 ; 0835 ; 1405 ;
摘要：
This paper addresses the problem of understanding joint attention in third-person social scene videos. Joint attention is the shared gaze behaviour of two or more individuals on an object or an area of interest and has a wide range of applications such as human-computer interaction, educational assessment, treatment of patients with attention disorders, and many more. Our method, Attention Flow, learns joint attention in an end-to-end fashion by using saliency-augmented attention maps and two novel convolutional attention mechanisms that determine to select relevant features and improve joint attention localization. We compare the effect of saliency maps and attention mechanisms and report quantitative and qualitative results on the detection and localization of joint attention in the VideoCoAtt dataset, which contains complex social scenes.
引用
收藏
页码：3316 / 3325
页数：10
相关论文
共 50 条
  • [21] Adversarial joint training with self-attention mechanism for robust end-to-end speech recognition
    Lujun Li
    Yikai Kang
    Yuchen Shi
    Ludwig Kürzinger
    Tobias Watzel
    Gerhard Rigoll
    EURASIP Journal on Audio, Speech, and Music Processing, 2021
  • [22] An End-to-End Heart Rate Estimation Scheme Using Divided Space-Time Attention
    Xin Zhang
    Changqiang Yang
    Ruonan Yin
    Lingzhuang Meng
    Neural Processing Letters, 2023, 55 : 2661 - 2685
  • [23] An End-to-End Heart Rate Estimation Scheme Using Divided Space-Time Attention
    Zhang, Xin
    Yang, Changqiang
    Yin, Ruonan
    Meng, Lingzhuang
    NEURAL PROCESSING LETTERS, 2023, 55 (03) : 2661 - 2685
  • [24] End-To-End Deep Learning Architecture for Continuous Blood Pressure Estimation Using Attention Mechanism
    Eom, Heesang
    Lee, Dongseok
    Han, Seungwoo
    Hariyani, Yuli Sun
    Lim, Yonggyu
    Sohn, Illsoo
    Park, Kwangsuk
    Park, Cheolsoo
    SENSORS, 2020, 20 (08)
  • [25] Attention-based end-to-end image defogging network
    Yang, Yan
    Zhang, Chen
    Jiang, Peipei
    Yue, Hui
    ELECTRONICS LETTERS, 2020, 56 (15) : 759 - +
  • [26] An End-to-end Speech Recognition Algorithm based on Attention Mechanism
    Chen, Jia-nan
    Gao, Shuang
    Sun, Han-zhe
    Liu, Xiao-hui
    Wang, Zi-ning
    Zheng, Yan
    PROCEEDINGS OF THE 39TH CHINESE CONTROL CONFERENCE, 2020, : 2935 - 2940
  • [27] Gated End-to-End Memory Network Based on Attention Mechanism
    Zhou, Bin
    Dang, Xin
    2018 INTERNATIONAL CONFERENCE ON ORANGE TECHNOLOGIES (ICOT), 2018,
  • [28] Hybrid CTC/Attention Architecture for End-to-End Speech Recognition
    Watanabe, Shinji
    Hori, Takaaki
    Kim, Suyoun
    Hershey, John R.
    Hayashi, Tomoki
    IEEE JOURNAL OF SELECTED TOPICS IN SIGNAL PROCESSING, 2017, 11 (08) : 1240 - 1253
  • [29] NEAT: Neural Attention Fields for End-to-End Autonomous Driving
    Chitta, Kashyap
    Prakash, Aditya
    Geiger, Andreas
    2021 IEEE/CVF INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV 2021), 2021, : 15773 - 15783
  • [30] Multi-channel Attention for End-to-End Speech Recognition
    Braun, Stefan
    Neil, Daniel
    Anumula, Jithendar
    Ceolini, Enea
    Liu, Shih-Chii
    19TH ANNUAL CONFERENCE OF THE INTERNATIONAL SPEECH COMMUNICATION ASSOCIATION (INTERSPEECH 2018), VOLS 1-6: SPEECH RESEARCH FOR EMERGING MARKETS IN MULTILINGUAL SOCIETIES, 2018, : 17 - 21
12345