A parallel variable memory BFGS training algorithm

被引：0
|
作者：
Mc Loone, S [1 ]
机构：
[1] Queens Univ Belfast, Sch Elect & Elect Engn, Intelligent Syst & Control Res Grp, Belfast BT9 5AH, Antrim, North Ireland
来源：
ALGORITHMS AND ARCHITECTURES FOR REAL-TIME CONTROL 2000 | 2000年
关键词：
neural networks; parallel algorithms; training; second-order;
D O I：
暂无
中图分类号：
TP [自动化技术、计算机技术];
学科分类号：
0812 ;
摘要：
This paper considers the parallel implementation of a novel variable memory quasi-newton neural network training algorithm recently developed by the author. Unlike existing training methods this new technique is able to optimize performance in relation to available memory. Numerically it has equivalent properties to Full Memory BFGS optimization (FM) when there are no restrictions on memory and to FM with periodic reset when memory is limited. Parallel implementations of both the Full and Variable Memory BFGS algorithms are outlined and performance results presented for a PVM target architecture. Copyright (C) 2000 IFAC.
引用
收藏
页码：129 / 134
页数：4
相关论文
共 50 条
  • [1] A memory optimal BFGS neural network training algorithm
    McLoone, SF
    Asirvadam, VS
    Irwin, GW
    PROCEEDING OF THE 2002 INTERNATIONAL JOINT CONFERENCE ON NEURAL NETWORKS, VOLS 1-3, 2002, : 513 - 518
  • [2] A New Stochastic Limited Memory BFGS Algorithm
    Momeni, M.
    Peyghami, M. R.
    Tarzanagh, D. Ataee
    JOURNAL OF MATHEMATICAL EXTENSION, 2020, 14 (03) : 65 - 83
  • [3] A variable memory Quasi-Newton training algorithm
    McLoone, S
    Irwin, G
    NEURAL PROCESSING LETTERS, 1999, 9 (01) : 77 - 89
  • [4] A Variable Memory Quasi-Newton Training Algorithm
    Seán McLoone
    George Irwin
    Neural Processing Letters, 1999, 9 : 77 - 89
  • [5] A memoryless BFGS neural network training algorithm
    Apostolopoulou, M. S.
    Sotiropoulos, D. C.
    Livieris, I. E.
    Pintelas, P.
    2009 7TH IEEE INTERNATIONAL CONFERENCE ON INDUSTRIAL INFORMATICS, VOLS 1 AND 2, 2009, : 216 - +
  • [6] An Adaptive Memory Multi-Batch L-BFGS Algorithm for Neural Network Training
    Zocco, Federico
    McLoone, Sean
    IFAC PAPERSONLINE, 2020, 53 (02): : 8199 - 8204
  • [7] Parallel L-BFGS-B algorithm on GPU
    Fei, Yun
    Rong, Guodong
    Wang, Bin
    Wang, Wenping
    COMPUTERS & GRAPHICS-UK, 2014, 40 : 1 - 9
  • [8] The BFGS parallel algorithm of unconstrained optimization problems in distributed environment
    Li, Wenjing
    Chen, Binglian
    Guo, Qingping
    DCABES 2006 PROCEEDINGS, VOLS 1 AND 2, 2006, : 18 - 23
  • [9] An Active Set Limited Memory BFGS Algorithm for Machine Learning
    Liu, Hanger
    Li, Yan
    Zhang, Maojun
    SYMMETRY-BASEL, 2022, 14 (02):
  • [10] A Stochastic Modified Limited Memory BFGS for Training Deep Neural Networks
    Yousefi, Mahsa
    Calomardo, Angeles Martinez
    INTELLIGENT COMPUTING, VOL 2, 2022, 507 : 9 - 28
12345