An improved Adagrad gradient descent optimization algorithm

被引：0

作者：

Zhang, N. ^{[1
]}

Lei, D. ^{[1
]}

Zhao, J. F. ^{[1
]}

机构：

[1] Shanghai Univ, Sch Mech Engn & Automat, Shanghai, Peoples R China

来源：

2018 CHINESE AUTOMATION CONGRESS (CAC) | 2018年

关键词：

deep learning; gradient descent; convergence; overfitting;

D O I：

暂无

中图分类号：

TP [自动化技术、计算机技术];

学科分类号：

0812 ;

摘要：

Gradient descent optimization algorithm is very important in deep learning. In order to obtain a more stable convergence process and reduce overfitting in multiple epochs, we propose an improved Adagrad gradient descent optimization algorithm in this paper. Our approach is tested both on the Reuters dataset and the IMDB dataset with many gradient descent optimization algorithms. The results show that our approach has a more stable convergence process and can reduce overfitting in multiple epochs.

引用

页码：2359 / 2362

页数：4

共 50 条

[11] A Descent Generalized RMIL Spectral Gradient Algorithm for Optimization Problems
Sulaiman, Ibrahim M.
Kaelo, P.
Khalid, Ruzelan
Nawawi, Mohd Kamal M.
INTERNATIONAL JOURNAL OF APPLIED MATHEMATICS AND COMPUTER SCIENCE, 2024, 34 (02) : 225 - 233
[12] Improvement of SPGD by Gradient Descent Optimization Algorithm in Deep Learning
Zhao, Qingsong
Hao, Shiqi
Wang, Yong
Wang, Lei
Lin, Zhi
2022 ASIA COMMUNICATIONS AND PHOTONICS CONFERENCE, ACP, 2022, : 469 - 472
[13] Multiple-gradient descent algorithm (MGDA) for multiobjective optimization
Desideri, Jean-Antoine
COMPTES RENDUS MATHEMATIQUE, 2012, 350 (5-6) : 313 - 318
[14] An improved stochastic gradient descent algorithm based on Renyi differential privacy
Cheng, XianFu
Yao, YanQing
Zhang, Liying
Liu, Ao
Li, Zhoujun
INTERNATIONAL JOURNAL OF INTELLIGENT SYSTEMS, 2022, 37 (12) : 10694 - 10714
[15] Improved clustering and anisotropic gradient descent algorithm for compact RBF network
Zeng, Delu
Xie, Shengli
Zhou, Zhiheng
NEURAL INFORMATION PROCESSING, PT 2, PROCEEDINGS, 2006, 4233 : 806 - 813
[16] Improved method of stochastic parallel gradient descent algorithm with global coupling
Jiang, Pengzhi
Liang, Yonghui
Xu, Jieping
Mao, Hongjun
Guangxue Xuebao/Acta Optica Sinica, 2014, 34
[17] Sequential convergence of AdaGrad algorithm for smooth convex optimization
Traore, Cheik
Pauwels, Edouard
OPERATIONS RESEARCH LETTERS, 2021, 49 (04) : 452 - 458
[18] Improved specular point prediction precision using gradient descent algorithm
Tian, Yusen
Xia, Junming
Sun, Yueqiang
Wang, Xianyi
Du, Qifei
Bai, Weihua
Wang, Dongwei
Cai, Yuerong
Wu, Chunjun
Li, Fu
Qiao, Hao
ADVANCES IN SPACE RESEARCH, 2020, 65 (06) : 1568 - 1579
[19] Traffic Signal Timings Optimization Based on Genetic Algorithm and Gradient Descent
Yadav, Alok
Nuthong, Chaiwat
2020 5TH INTERNATIONAL CONFERENCE ON COMPUTER AND COMMUNICATION SYSTEMS (ICCCS 2020), 2020, : 670 - 674
[20] Iterative quantum algorithm for combinatorial optimization based on quantum gradient descent
Yi, Xin
Huo, Jia-Cheng
Gao, Yong-Pan
Fan, Ling
Zhang, Ru
Cao, Cong
RESULTS IN PHYSICS, 2024, 56

← 1 2 3 4 5 →