Manifold Regularized Deep Neural Networks

被引:0
|
作者
Tomar, Vikrant Singh [1 ]
Rose, Richard C. [1 ]
机构
[1] McGill Univ, Dept Elect & Comp Engn, Montreal, PQ, Canada
关键词
manifold learning; deep neural networks; speech recognition; tandem feature extraction;
D O I
暂无
中图分类号
TP18 [人工智能理论];
学科分类号
081104 ; 0812 ; 0835 ; 1405 ;
摘要
Deep neural networks (DNNs) have been successfully applied to a variety of automatic speech recognition (ASR) tasks, both in discriminative feature extraction and hybrid acoustic modeling scenarios. The development of improved loss functions and regularization approaches have resulted in consistent reductions in ASR word error rates (WERs). This paper presents a manifold learning based regularization framework for DNN training. The associated techniques attempt to preserve the underlying low dimensional manifold based relationships amongst speech feature vectors as part of the optimization procedure for estimating network parameters. This is achieved by imposing manifold based locality preserving constraints on the outputs of the network. The techniques are presented in the context of a bottleneck DNN architecture for feature extraction in a tandem configuration. The ASR WER obtained using these networks is evaluated on a speech-in-noise task and compared to that obtained using DNN-bottleneck networks trained without manifold constraints.
引用
收藏
页码:348 / 352
页数:5
相关论文
共 50 条
  • [21] Analytic Function Approximation by Path-Norm-Regularized Deep Neural Networks
    Beknazaryan, Aleksandr
    [J]. ENTROPY, 2022, 24 (08)
  • [22] Exploiting Feature and Class Relationships in Video Categorization with Regularized Deep Neural Networks
    Jiang, Yu-Gang
    Wu, Zuxuan
    Wang, Jun
    Xue, Xiangyang
    Chang, Shih-Fu
    [J]. IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, 2018, 40 (02) : 352 - 364
  • [23] L2 regularized deep convolutional neural networks for fire detection
    Roy, Sanjiban Sekhar
    Goti, Vatsal
    Sood, Aditya
    Roy, Harsh
    Gavrila, Tania
    Floroian, Dan
    Paraschiv, Nicolae
    Mohammadi-Ivatloo, Behnam
    [J]. JOURNAL OF INTELLIGENT & FUZZY SYSTEMS, 2022, 43 (02) : 1799 - 1810
  • [24] Convergent Block Coordinate Descent for Training Tikhonov Regularized Deep Neural Networks
    Zhang, Ziming
    Brand, Matthew
    [J]. ADVANCES IN NEURAL INFORMATION PROCESSING SYSTEMS 30 (NIPS 2017), 2017, 30
  • [25] Dual-regularized matrix factorization with deep neural networks for recommender systems
    Wu, Hao
    Zhang, Zhengxin
    Yue, Kun
    Zhang, Binbin
    He, Jun
    Sun, Liangchen
    [J]. KNOWLEDGE-BASED SYSTEMS, 2018, 145 : 46 - 58
  • [26] Transfer learning with deep manifold regularized auto-encoders
    Zhu, Yi
    Wu, Xindong
    Li, Peipei
    Zhang, Yuhong
    Hu, Xuegang
    [J]. NEUROCOMPUTING, 2019, 369 : 145 - 154
  • [27] Stability to Deformations of Manifold Filters and Manifold Neural Networks
    Wang, Zhiyang
    Ruiz, Luana
    Ribeiro, Alejandro
    [J]. IEEE TRANSACTIONS ON SIGNAL PROCESSING, 2024, 72 : 2130 - 2146
  • [28] Statistical guarantees for regularized neural networks
    Taheri, Mahsa
    Xie, Fang
    Lederer, Johannes
    [J]. NEURAL NETWORKS, 2021, 142 : 148 - 161
  • [29] Automated diagnostics for manufacturing machinery based on well-regularized deep neural networks
    DiBiano, Robert
    Mukhopadhyay, Supratik
    [J]. INTEGRATION-THE VLSI JOURNAL, 2017, 58 : 303 - 310
  • [30] Optimizing Solar Power Using Array Topology Reconfiguration With Regularized Deep Neural Networks
    Narayanaswamy, Vivek
    Ayyanar, Raja
    Tepedelenlioglu, Cihan
    Srinivasan, Devarajan
    Spanias, Andreas
    [J]. IEEE ACCESS, 2023, 11 : 7461 - 7470