Learning with regularizers in multilayer neural networks

被引：8

作者：

Saad, D ^{[1
]}

Rattray, M ^{[1
]}

机构：

[1] Aston Univ, Dept Comp Sci & Appl Math, Birmingham B4 7ET, W Midlands, England

来源：

PHYSICAL REVIEW E | 1998年 / 57卷 / 02期

关键词：

D O I：

10.1103/PhysRevE.57.2170

中图分类号：

O35 [流体力学]; O53 [等离子体物理学];

学科分类号：

070204 ; 080103 ; 080704 ;

摘要：

We study the effect of regularization in an on-line gradient-descent learning scenario for a general two-layer student network with an arbitrary number of hidden units. Training examples are randomly drawn input vectors labeled by a two-layer teacher network with an arbitrary number of hidden units that may be corrupted by Gaussian output noise. We examine the effect of weight decay regularization on the dynamical evolution of the order parameters and generalization error in various phases of the learning process, in both noiseless and noisy scenarios.

引用

页码：2170 / 2176

页数：7

共 50 条

[1] Learning with noise and regularizers in multilayer neural networks
Saad, D
Solla, SA
[J]. ADVANCES IN NEURAL INFORMATION PROCESSING SYSTEMS 9: PROCEEDINGS OF THE 1996 CONFERENCE, 1997, 9 : 260 - 266
[2] Fault Tolerant Regularizers for Multilayer Feedforward Networks
Qiao, Deng-yu
Leung, Chi Sing
Sum, Pui Fai
[J]. NEURAL INFORMATION PROCESSING, PT 1, PROCEEDINGS, 2009, 5863 : 277 - +
[3] Regularizers for fault tolerant multilayer feedforward networks
Mak, Shue Kwan
Sum, Pui-Fai
Leung, Chi-Sing
[J]. NEUROCOMPUTING, 2011, 74 (11) : 2028 - 2040
[4] Optimal learning in multilayer neural networks
Winther, O
Lautrup, B
Zhang, JB
[J]. PHYSICAL REVIEW E, 1997, 55 (01) : 836 - 844
[5] LEARNING ALGORITHMS FOR MULTILAYER NEURAL NETWORKS
AVEDYAN, ED
[J]. AUTOMATION AND REMOTE CONTROL, 1995, 56 (04) : 541 - 551
[6] Regression with Deep Neural Networks: Generalization Error Guarantees, Learning Algorithms, and Regularizers
Amjad, Jaweria
Lyu, Zhaoyan
Rodrigues, Miguel R. D.
[J]. 29TH EUROPEAN SIGNAL PROCESSING CONFERENCE (EUSIPCO 2021), 2021, : 1481 - 1485
[7] Quantizability and learning complexity in multilayer neural networks
Fu, LM
[J]. IEEE TRANSACTIONS ON SYSTEMS MAN AND CYBERNETICS PART C-APPLICATIONS AND REVIEWS, 1998, 28 (02): : 295 - 300
[8] An algorithm of supervised learning for multilayer neural networks
Tang, Z
Wang, XG
Tamura, H
Ishii, M
[J]. NEURAL COMPUTATION, 2003, 15 (05) : 1125 - 1142
[9] Supervised Learning in Multilayer Spiking Neural Networks
Sporea, Ioana
Gruening, Andre
[J]. NEURAL COMPUTATION, 2013, 25 (02) : 473 - 509
[10] Effect of batch learning in multilayer neural networks
Fukumizu, K
[J]. ICONIP'98: THE FIFTH INTERNATIONAL CONFERENCE ON NEURAL INFORMATION PROCESSING JOINTLY WITH JNNS'98: THE 1998 ANNUAL CONFERENCE OF THE JAPANESE NEURAL NETWORK SOCIETY - PROCEEDINGS, VOLS 1-3, 1998, : 67 - 70

← 1 2 3 4 5 →