Thank you for attention: A survey on attention-based artificial neural networks for automatic speech recognition

被引：0
|
作者：
Karmakar, Priyabrata [1 ]
Teng, Shyh Wei [1 ]
Lu, Guojun [2 ]
机构：
[1] Federat Univ, Inst Innovat Sci & Sustainabil, Ballarat, Australia
[2] Federat Univ, Global Profess Sch, Ballarat, Australia
来源：
INTELLIGENT SYSTEMS WITH APPLICATIONS | 2024年 / 23卷
关键词：
Automatic speech recognition (ASR); Attention mechanism; Recurrent neural network (RNN); Transformer; Offline ASR; Streaming ASR; SELF-ATTENTION;
D O I：
10.1016/j.iswa.2024.200406
中图分类号：
TP18 [人工智能理论];
学科分类号：
081104 ; 0812 ; 0835 ; 1405 ;
摘要：
Attention is a very popular and effective mechanism in artificial neural network-based sequence-to-sequence models. In this survey paper, a comprehensive review of the different attention models used in developing automatic speech recognition systems is provided. The paper focuses on how attention models have grown and changed for offline and streaming speech recognition in recurrent neural networks and Transformer-based systems.
引用
收藏
页数：12
相关论文
共 50 条
  • [1] Attention-based graph neural networks: a survey
    Chengcheng Sun
    Chenhao Li
    Xiang Lin
    Tianji Zheng
    Fanrong Meng
    Xiaobin Rui
    Zhixiao Wang
    [J]. Artificial Intelligence Review, 2023, 56 : 2263 - 2310
  • [2] Attention-based graph neural networks: a survey
    Sun, Chengcheng
    Li, Chenhao
    Lin, Xiang
    Zheng, Tianji
    Meng, Fanrong
    Rui, Xiaobin
    Wang, Zhixiao
    [J]. ARTIFICIAL INTELLIGENCE REVIEW, 2023, 56 (SUPPL 2) : 2263 - 2310
  • [3] Attention-Based Models for Speech Recognition
    Chorowski, Jan
    Bahdanau, Dzmitry
    Serdyuk, Dmitriy
    Cho, Kyunghyun
    Bengio, Yoshua
    [J]. ADVANCES IN NEURAL INFORMATION PROCESSING SYSTEMS 28 (NIPS 2015), 2015, 28
  • [4] Attention-Based Bimodal Neural Network Speech Recognition System on FPGA
    Chen, Aiwu
    [J]. Informatica (Slovenia), 2025, 49 (13): : 1 - 12
  • [5] An Online Attention-Based Model for Speech Recognition
    Fan, Ruchao
    Zhou, Pan
    Chen, Wei
    Jia, Jia
    Liu, Gang
    [J]. INTERSPEECH 2019, 2019, : 4390 - 4394
  • [6] Attention-based Audio-Visual Fusion for Robust Automatic Speech Recognition
    Sterpu, George
    Saam, Christian
    Harte, Naomi
    [J]. ICMI'18: PROCEEDINGS OF THE 20TH ACM INTERNATIONAL CONFERENCE ON MULTIMODAL INTERACTION, 2018, : 111 - 115
  • [7] Attention-Based Radar PRI Modulation Recognition With Recurrent Neural Networks
    Li, Xueqiong
    Liu, Zhangmeng
    Huang, Zhitao
    [J]. IEEE ACCESS, 2020, 8 (08): : 57426 - 57436
  • [8] An Attention-Based Convolutional Recurrent Neural Networks for Scene Text Recognition
    Alshawi, Adil Abdullah Abdulhussein
    Tanha, Jafar
    Balafar, Mohammad Ali
    [J]. IEEE ACCESS, 2024, 12 : 8123 - 8134
  • [9] A Neural Autoregressive Approach to Attention-based Recognition
    Zheng, Yin
    Zemel, Richard S.
    Zhang, Yu-Jin
    Larochelle, Hugo
    [J]. INTERNATIONAL JOURNAL OF COMPUTER VISION, 2015, 113 (01) : 67 - 79
  • [10] A Neural Autoregressive Approach to Attention-based Recognition
    Yin Zheng
    Richard S. Zemel
    Yu-Jin Zhang
    Hugo Larochelle
    [J]. International Journal of Computer Vision, 2015, 113 : 67 - 79
12345