On the optimality of the Oja's algorithm for online PCA

被引：3
|
作者：
Liang, Xin [1 ,2 ]
机构：
[1] Tsinghua Univ, Yau Math Sci Ctr, Beijing 100084, Peoples R China
[2] Yanqi Lake Beijing Inst Math Sci & Applicat, Beijing 101408, Peoples R China
来源：
STATISTICS AND COMPUTING | 2023年 / 33卷 / 03期
基金：
中国国家自然科学基金;
关键词：
Principal component analysis; Stochastic approximation; High-dimensional data; Oja's algorithm; STOCHASTIC-APPROXIMATION; PRINCIPAL;
D O I：
10.1007/s11222-023-10229-z
中图分类号：
TP301 [理论、方法];
学科分类号：
081202 ;
摘要：
In this paper we analyze the behavior of the Oja's algorithm for online/streaming principal component subspace estimation. It is proved that with high probability it performs an efficient, gap-free, global convergence rate to approximate an principal component subspace for any sub-Gaussian distribution. Moreover, it is the first time to show that the convergence rate, namely the upper bound of the approximation, exactly matches the lower bound of an approximation obtained by the offline/classical PCA up to a constant factor.
引用
收藏
页数：11
相关论文
共 50 条
  • [11] Global convergence of Oja's PCA learning algorithm with a non-zero-approaching adaptive learning rate
    Lv, Jian Cheng
    Yi, Zhang
    Tan, K. K.
    THEORETICAL COMPUTER SCIENCE, 2006, 367 (03) : 286 - 307
  • [12] Streaming k-PCA: Efficient guarantees for Oja’s algorithm, beyond rank-one updates
    Huang, De
    Niles-Weed, Jonathan
    Ward, Rachel
    Proceedings of Machine Learning Research, 2021, 134 : 2463 - 2498
  • [13] Streaming k-PCA: Efficient guarantees for Oja's algorithm, beyond rank-one updates
    Huang, De
    Niles-Weed, Jonathan
    Ward, Rachel
    CONFERENCE ON LEARNING THEORY, VOL 134, 2021, 134
  • [14] On stability of Oja algorithm
    Institute of Mathematics, Polish Academy of Sciences, Poland
    不详
    Lect. Notes Comput. Sci., 1600, (354-360):
  • [15] Orthogonal Oja algorithm
    Abed-Meraim, K
    Attallah, S
    Chkeif, A
    Hua, Y
    IEEE SIGNAL PROCESSING LETTERS, 2000, 7 (05) : 116 - 119
  • [16] ODE-Inspired Analysis for the Biological Version of Oja's Rule in Solving Streaming PCA
    Chou, Chi-Ning
    Wang, Mien Brabeeba
    CONFERENCE ON LEARNING THEORY, VOL 125, 2020, 125
  • [17] On the Optimal Tradeoff Between Computational Efficiency and Generalizability of Oja's Algorithm
    Xu, Xiangxiang
    Huang, Shao-Lun
    IEEE ACCESS, 2020, 8 : 102616 - 102628
  • [18] Subspace estimation and tracking using enhanced versions of Oja's algorithm
    Attallah, S
    Abed-Meraim, K
    2001 IEEE THIRD WORKSHOP ON SIGNAL PROCESSING ADVANCES IN WIRELESS COMMUNICATIONS, PROCEEDINGS, 2001, : 218 - 220
  • [19] Adaptivity and Optimality: A Universal Algorithm for Online Convex Optimization
    Wang, Guanghui
    Lu, Shiyin
    Zhang, Lijun
    35TH UNCERTAINTY IN ARTIFICIAL INTELLIGENCE CONFERENCE (UAI 2019), 2020, 115 : 659 - 668
  • [20] Global convergence of Oja's subspace algorithm for principal component extraction
    Chen, TP
    Hua, YB
    Yan, WY
    IEEE TRANSACTIONS ON NEURAL NETWORKS, 1998, 9 (01): : 58 - 67
12345