Globally optimal parameters for on-line learning in multilayer neural networks

被引：31
|
作者：
Saad, D
Rattray, M
机构：
[1] Department of Computer Science and Applied Mathematics, Aston University, Birmingham
来源：
PHYSICAL REVIEW LETTERS | 1997年 / 79卷 / 13期
关键词：
D O I：
10.1103/PhysRevLett.79.2578
中图分类号：
O4 [物理学];
学科分类号：
0702 ;
摘要：
We present a framework for calculating globally optimal parameters, within a given time frame, for on-line learning in multilayer neural networks. We demonstrate the capability of this method by computing optimal learning rates in typical learning scenarios. A similar treatment allows one to determine the relevance of related training algorithms based on modifications to the basic gradient descent rule as well as to compare different training methods.
引用
收藏
页码：2578 / 2581
页数：4
相关论文
共 50 条
  • [1] Globally optimal learning rates for multilayer neural networks
    Saad, D
    Rattray, M
    [J]. PHILOSOPHICAL MAGAZINE B-PHYSICS OF CONDENSED MATTER STATISTICAL MECHANICS ELECTRONIC OPTICAL AND MAGNETIC PROPERTIES, 1998, 77 (05): : 1523 - 1530
  • [2] Globally optimal on-line learning rules for multi-layer neural networks
    Rattray, M
    Saad, D
    [J]. JOURNAL OF PHYSICS A-MATHEMATICAL AND GENERAL, 1997, 30 (22): : L771 - L776
  • [3] Globally optimal on-line learning rules
    Rattray, M
    Saad, D
    [J]. ADVANCES IN NEURAL INFORMATION PROCESSING SYSTEMS 10, 1998, 10 : 322 - 328
  • [4] Dynamics of on-line gradient descent learning for multilayer neural networks
    Saad, D
    Solla, SA
    [J]. ADVANCES IN NEURAL INFORMATION PROCESSING SYSTEMS 8: PROCEEDINGS OF THE 1995 CONFERENCE, 1996, 8 : 302 - 308
  • [5] Matrix-update for accelerated on-line learning in multilayer neural networks
    Bos, S
    [J]. JOURNAL OF PHYSICS A-MATHEMATICAL AND GENERAL, 1998, 31 (22): : L413 - L417
  • [6] Finite-size effects in on-line learning of multilayer neural networks
    Barber, D
    Saad, D
    Sollich, P
    [J]. EUROPHYSICS LETTERS, 1996, 34 (02): : 151 - 156
  • [7] On-line learning from restricted training sets in multilayer neural networks
    Coolen, ACC
    Saad, D
    Xiong, YS
    [J]. EUROPHYSICS LETTERS, 2000, 51 (06): : 691 - 697
  • [8] Optimal learning in multilayer neural networks
    Winther, O
    Lautrup, B
    Zhang, JB
    [J]. PHYSICAL REVIEW E, 1997, 55 (01) : 836 - 844
  • [9] Dynamic multilayer neural networks for nonlinear system on-line identification
    Yu, W
    Poznyak, AS
    Sanchez, EN
    [J]. PROCEEDINGS OF THE 2000 IEEE INTERNATIONAL SYMPOSIUM ON INTELLIGENT CONTROL, 2000, : 25 - 30
  • [10] Multilayer neural networks:: an experimental evaluation of on-line training methods
    Martí, R
    El-Fallahi, A
    [J]. COMPUTERS & OPERATIONS RESEARCH, 2004, 31 (09) : 1491 - 1513
12345