An improved Adagrad gradient descent optimization algorithm

被引：0

作者：

Zhang, N. ^{[1
]}

Lei, D. ^{[1
]}

Zhao, J. F. ^{[1
]}

机构：

[1] Shanghai Univ, Sch Mech Engn & Automat, Shanghai, Peoples R China

来源：

2018 CHINESE AUTOMATION CONGRESS (CAC) | 2018年

关键词：

deep learning; gradient descent; convergence; overfitting;

D O I：

暂无

中图分类号：

TP [自动化技术、计算机技术];

学科分类号：

0812 ;

摘要：

Gradient descent optimization algorithm is very important in deep learning. In order to obtain a more stable convergence process and reduce overfitting in multiple epochs, we propose an improved Adagrad gradient descent optimization algorithm in this paper. Our approach is tested both on the Reuters dataset and the IMDB dataset with many gradient descent optimization algorithms. The results show that our approach has a more stable convergence process and can reduce overfitting in multiple epochs.

引用

页码：2359 / 2362

页数：4

共 50 条

[1] Adaptive gradient descent optimization algorithm with improved differential term
Ge, Quan-Bo
Zhang, Jian-Chao
Yang, Qin-Min
Li, Hong
Kongzhi Lilun Yu Yingyong/Control Theory and Applications, 2022, 39 (04): : 623 - 632
[2] AdaGrad Gradient Descent Method for AI Image Management
Fang, Jen-Kuang
Fong, Cher-Min
Yang, Peng
Hung, Chao-Kai
Lu, Wen-Long
Chang, Chien-Wei
2020 IEEE INTERNATIONAL CONFERENCE ON CONSUMER ELECTRONICS - TAIWAN (ICCE-TAIWAN), 2020,
[3] An improved adaptive momentum gradient descent algorithm
Jiang Z.
Song J.
Liu Y.
Huazhong Keji Daxue Xuebao (Ziran Kexue Ban)/Journal of Huazhong University of Science and Technology (Natural Science Edition), 2023, 51 (05): : 137 - 143
[4] The Improved Stochastic Fractional Order Gradient Descent Algorithm
Yang, Yang
Mo, Lipo
Hu, Yusen
Long, Fei
FRACTAL AND FRACTIONAL, 2023, 7 (08)
[5] Quantized Gradient Descent Algorithm for Distributed Nonconvex Optimization
Yoshida, Junya
Hayashi, Naoki
Takai, Shigemasa
IEICE TRANSACTIONS ON FUNDAMENTALS OF ELECTRONICS COMMUNICATIONS AND COMPUTER SCIENCES, 2023, E106A (10) : 1297 - 1304
[6] Smoothing gradient descent algorithm for the composite sparse optimization
Yang, Wei
Pan, Lili
Wan, Jinhui
AIMS MATHEMATICS, 2024, 9 (12): : 33401 - 33422
[7] An acceleration of gradient descent algorithm with backtracking for unconstrained optimization
Neculai Andrei
Numerical Algorithms, 2006, 42 : 63 - 73
[8] An acceleration of gradient descent algorithm with backtracking for unconstrained optimization
Andrei, Neculai
NUMERICAL ALGORITHMS, 2006, 42 (01) : 63 - 73
[9] Momentum-weighted conjugate gradient descent algorithm for gradient coil optimization
Lu, HB
Jesmanowicz, A
Li, SJ
Hyde, JS
MAGNETIC RESONANCE IN MEDICINE, 2004, 51 (01) : 158 - 164
[10] An RNA evolutionary algorithm based on gradient descent for function optimization
Wu, Qiuxuan
Zhao, Zikai
Chen, Mingming
Chi, Xiaoni
Zhang, Botao
Wang, Jian
Zhilenkov, Anton A.
Chepinskiy, Sergey A.
JOURNAL OF COMPUTATIONAL DESIGN AND ENGINEERING, 2024, 11 (04) : 332 - 357

← 1 2 3 4 5 →