Implicit Regularization in Matrix Factorization

被引：0

作者：

Gunasekar, Suriya

Woodworth, Blake

Bhojanapalli, Srinadh

Neyshabur, Behnam

Srebro, Nathan

机构：

来源：

2018 INFORMATION THEORY AND APPLICATIONS WORKSHOP (ITA) | 2018年

关键词：

RANK; NORM;

D O I：

暂无

中图分类号：

TP [自动化技术、计算机技术];

学科分类号：

0812 ;

摘要：

We study implicit regularization when optimizing an underdetermined quadratic objective over a matrix X with gradient descent on a factorization of X. We conjecture and provide empirical and theoretical evidence that with small enough step sizes and initialization close enough to the origin, gradient descent on a full dimensional factorization converges to the minimum nuclear norm solution.

引用

页数：10

共 50 条

[1] Implicit Regularization in Matrix Factorization
Gunasekar, Suriya
Woodworth, Blake
Bhojanapalli, Srinadh
Neyshabur, Behnam
Srebro, Nathan
[J]. ADVANCES IN NEURAL INFORMATION PROCESSING SYSTEMS 30 (NIPS 2017), 2017, 30
[2] Implicit Regularization in Deep Matrix Factorization
Arora, Sanjeev
Cohen, Nadav
Hu, Wei
Luo, Yuping
[J]. ADVANCES IN NEURAL INFORMATION PROCESSING SYSTEMS 32 (NIPS 2019), 2019, 32
[3] Implicit Regularization in Tensor Factorization
Razin, Noam
Maman, Asaf
Cohen, Nadav
[J]. INTERNATIONAL CONFERENCE ON MACHINE LEARNING, VOL 139, 2021, 139
[4] Implicit Regularization in Deep Tensor Factorization
Milanesi, Paolo
Kadri, Hachem
Ayache, Stephan
Artieres, Thierry
[J]. 2021 INTERNATIONAL JOINT CONFERENCE ON NEURAL NETWORKS (IJCNN), 2021,
[5] THE NONNEGATIVE MATRIX FACTORIZATION: REGULARIZATION AND COMPLEXITY
Ito, K.
Landi, A. K.
[J]. SIAM JOURNAL ON SCIENTIFIC COMPUTING, 2016, 38 (02): : B327 - B346
[6] Implicit Regularization with Polynomial Growth in Deep Tensor Factorization
Hariz, Kais
Kadri, Hachem
Ayache, Stephane
Moakher, Maher
Artieres, Thierry
[J]. INTERNATIONAL CONFERENCE ON MACHINE LEARNING, VOL 162, 2022,
[7] Adaptive and Implicit Regularization for Matrix Completion
Li, Zhemin
Sun, Tao
Wang, Hongxia
Wang, Bao
[J]. SIAM JOURNAL ON IMAGING SCIENCES, 2022, 15 (04): : 2000 - 2022
[8] Double Regularization Matrix Factorization Recommendation Algorithm
Du, Ruizhong
Lu, Jiaojiao
Cai, Hongyun
[J]. IEEE ACCESS, 2019, 7 : 139668 - 139677
[9] Convex nonnegative matrix factorization with manifold regularization
Hu, Wenjun
Choi, Kup-Sze
Wang, Peiliang
Jiang, Yunliang
Wang, Shitong
[J]. NEURAL NETWORKS, 2015, 63 : 94 - 103
[10] Personalized Regularization Learning for Fairer Matrix Factorization
Yao, Sirui
Huang, Bert
[J]. ADVANCES IN KNOWLEDGE DISCOVERY AND DATA MINING, PAKDD 2021, PT II, 2021, 12713 : 600 - 611

← 1 2 3 4 5 →