Learning gradients by a gradient descent algorithm

被引：22

作者：

Dong, Xuemei ^{[2
]}

Zhou, Ding-Xuan ^{[1
]}

机构：

[1] City Univ Hong Kong, Dept Math, Kowloon, Hong Kong, Peoples R China

[2] Beijing Univ Aeronaut & Astronaut, Dept Math, Beijing 100083, Peoples R China

来源：

JOURNAL OF MATHEMATICAL ANALYSIS AND APPLICATIONS | 2008年 / 341卷 / 02期

关键词：

learning algorithm; stochastic gradient descent; variable selection; reproducing kernel Hilbert space; error analysis;

D O I：

10.1016/j.jmaa.2007.10.044

中图分类号：

O29 [应用数学];

学科分类号：

070104 ;

摘要：

We propose a stochastic gradient descent algorithm for learning the gradient of a regression function from random samples of function values. This is a learning algorithm involving Mercer kernels. By a detailed analysis in reproducing kernel Hilbert spaces, we provide some error bounds to show that the gradient estimated by the algorithm converges to the true gradient, under some natural conditions on the regression function and suitable choices of the step size and regularization parameters. (C) 2007 Elsevier Inc. All rights reserved.

引用

页码：1018 / 1027

页数：10

共 50 条

[31] A Modified Stochastic Gradient Descent Optimization Algorithm With Random Learning Rate for Machine Learning and Deep Learning
Shim, Duk-Sun
Shim, Joseph
[J]. INTERNATIONAL JOURNAL OF CONTROL AUTOMATION AND SYSTEMS, 2023, 21 (11) : 3825 - 3831
[32] Theoretical analysis of learning speed in gradient descent algorithm replacing derivative with constant
Hara, Kazuyuki
Katahira, Kentaro
[J]. IPSJ Online Transactions, 2014, 7 (01) : 14 - 19
[33] Efficient gradient descent algorithm for sparse models with application in learning-to-rank
Lai, Hanjiang
Pan, Yan
Tang, Yong
Liu, Ning
[J]. KNOWLEDGE-BASED SYSTEMS, 2013, 49 : 190 - 198
[34] GRADIENT DESCENT LEARNING ALGORITHM OVERVIEW - A GENERAL DYNAMICAL-SYSTEMS PERSPECTIVE
BALDI, P
[J]. IEEE TRANSACTIONS ON NEURAL NETWORKS, 1995, 6 (01): : 182 - 195
[35] Soft-Sign Stochastic Gradient Descent Algorithm for Wireless Federated Learning
Lee, Seunghoon
Park, Chanho
Hong, Songnam
Eldar, Yonina C.
Lee, Namyoon
[J]. SPAWC 2021: 2021 IEEE 22ND INTERNATIONAL WORKSHOP ON SIGNAL PROCESSING ADVANCES IN WIRELESS COMMUNICATIONS (IEEE SPAWC 2021), 2020, : 241 - 245
[36] Regularization Effect of Random Node Fault/Noise on Gradient Descent Learning Algorithm
Sum, John
Leung, Chi-Sing
[J]. IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS, 2023, 34 (05) : 2619 - 2632
[37] Study on adaptive fuzzy control system based on gradient descent learning algorithm
College of Electronic Information Eng., Chongqing University of Science and Technology, Chongqing 400050, China
不详
[J]. Xitong Fangzhen Xuebao, 2007, 6 (1265-1268+1273):
[38] Study on adaptive fuzzy control system based on gradient descent learning algorithm
Xiong, Jundi
Li, Taifu
Xiao, Huihui
Deng, Renming
[J]. FUZZY INFORMATION AND ENGINEERING, PROCEEDINGS, 2007, 40 : 1009 - +
[39] An efficient, distributed stochastic gradient descent algorithm for deep-learning applications
Cong, Guojing
Bhardwaj, Onkar
Feng, Minwei
[J]. 2017 46TH INTERNATIONAL CONFERENCE ON PARALLEL PROCESSING (ICPP), 2017, : 11 - 20
[40] Optimizing Stochastic Gradient Descent Using the Angle Between Gradients
Song, Chongya
Pons, Alexander
Yen, Kang
[J]. 2020 IEEE INTERNATIONAL CONFERENCE ON BIG DATA (BIG DATA), 2020, : 5269 - 5275

← 1 2 3 4 5 →