The Parallel Modification to the Levenberg-Marquardt Algorithm

被引：9

作者：

Bilski, Jaroslaw ^{[1
]}

Kowalczyk, Bartosz ^{[1
]}

Grzanek, Konrad ^{[2
,3
]}

机构：

[1] Czestochowa Tech Univ, Inst Computat Intelligence, Czestochowa, Poland

[2] Univ Social Sci, Informat Technol Inst, Lodz, Poland

[3] Clark Univ, Worcester, MA 01610 USA

来源：

ARTIFICIAL INTELLIGENCE AND SOFT COMPUTING, ICAISC 2018, PT I | 2018年 / 10841卷

关键词：

Feed-forward neural network; Parallel neural network training algorithm; Optimization problem; Levenberg-Marquardt algorithm; QR decomposition; Givens rotation;

D O I：

10.1007/978-3-319-91253-0_2

中图分类号：

TP18 [人工智能理论];

学科分类号：

081104 ; 0812 ; 0835 ; 1405 ;

摘要：

The paper presents a parallel approach to the Levenberg-Marquardt algorithm (also called LM or LMA). The first section contains the mathematical basics of the classic LMA. Then the parallel modification to LMA is introduced. The classic Levenberg-Marquardt algorithm is sufficient for a training of small neural networks. For bigger networks the algorithm complexity becomes too big for the effective teaching. The main scope of this paper is to propose more complexity efficient approach to LMA by parallel computation. The proposed modification to LMA has been tested on a few function approximation problems and has been compared to the classic LMA. The paper concludes with the resolution that the parallel modification to LMA could significantly improve algorithm performance for bigger networks. Summary also contains a several proposals for the possible future work directions in the considered area.

引用

页码：15 / 24

页数：10

共 50 条

[1] A Parallel Levenberg-Marquardt Algorithm
Cao, Jun
Novstrup, Krista A.
Goyal, Ayush
Midkiff, Samuel R.
Caruthers, James M.
[J]. ICS'09: PROCEEDINGS OF THE 2009 ACM SIGARCH INTERNATIONAL CONFERENCE ON SUPERCOMPUTING, 2009, : 450 - 459
[2] Parallel and separable recursive Levenberg-Marquardt training algorithm
Asirvadam, VS
McLoone, SF
Irwin, GW
[J]. NEURAL NETWORKS FOR SIGNAL PROCESSING XII, PROCEEDINGS, 2002, : 129 - 138
[3] Parallel Levenberg-Marquardt Algorithm Without Error Backpropagation
Bilski, Jaroslaw
Wilamowski, Bogdan M.
[J]. ARTIFICIAL INTELLIGENCE AND SOFT COMPUTING, ICAISC 2017, PT I, 2017, 10245 : 25 - 39
[4] Adaptive Levenberg-Marquardt Algorithm: A New Optimization Strategy for Levenberg-Marquardt Neural Networks
Yan, Zhiqi
Zhong, Shisheng
Lin, Lin
Cui, Zhiquan
[J]. MATHEMATICS, 2021, 9 (17)
[5] Parallel Approach to the Levenberg-Marquardt Learning Algorithm for Feedforward Neural Networks
Bilski, Jaroslaw
Smolag, Jacek
Zurada, Jacek M.
[J]. ARTIFICIAL INTELLIGENCE AND SOFT COMPUTING, PT I, 2015, 9119 : 3 - 14
[6] The application and modeling of the Levenberg-Marquardt algorithm
Li, Jian-rong
[J]. 2010 2ND INTERNATIONAL CONFERENCE ON E-BUSINESS AND INFORMATION SYSTEM SECURITY (EBISS 2010), 2010, : 278 - 280
[7] Distributed localization using Levenberg-Marquardt algorithm
Shervin Parvini Ahmadi
Anders Hansson
Sina Khoshfetrat Pakazad
[J]. EURASIP Journal on Advances in Signal Processing, 2021
[8] A Novel Modification on the Levenberg-Marquardt Algorithm for Avoiding Overfitting in Neural Network Training
Iplikci, Serdar
Bilgi, Batuhan
Menemen, Ali
Bahtiyar, Bedri
[J]. ARTIFICIAL NEURAL NETWORKS AND MACHINE LEARNING - ICANN 2019: DEEP LEARNING, PT II, 2019, 11728 : 201 - 207
[9] A Levenberg-Marquardt algorithm for unconstrained multicriteria optimization
Fischer, Andreas
Shukla, Pradyumn K.
[J]. OPERATIONS RESEARCH LETTERS, 2008, 36 (05) : 643 - 646
[10] Convergence analysis of a subsampled Levenberg-Marquardt algorithm
Xing, Ganchen
Gu, Jian
Xiao, Xiantao
[J]. OPERATIONS RESEARCH LETTERS, 2023, 51 (04) : 379 - 384

← 1 2 3 4 5 →