On the optimality of the Oja's algorithm for online PCA

被引：3
|
作者：
Liang, Xin [1 ,2 ]
机构：
[1] Tsinghua Univ, Yau Math Sci Ctr, Beijing 100084, Peoples R China
[2] Yanqi Lake Beijing Inst Math Sci & Applicat, Beijing 101408, Peoples R China
来源：
STATISTICS AND COMPUTING | 2023年 / 33卷 / 03期
基金：
中国国家自然科学基金;
关键词：
Principal component analysis; Stochastic approximation; High-dimensional data; Oja's algorithm; STOCHASTIC-APPROXIMATION; PRINCIPAL;
D O I：
10.1007/s11222-023-10229-z
中图分类号：
TP301 [理论、方法];
学科分类号：
081202 ;
摘要：
In this paper we analyze the behavior of the Oja's algorithm for online/streaming principal component subspace estimation. It is proved that with high probability it performs an efficient, gap-free, global convergence rate to approximate an principal component subspace for any sub-Gaussian distribution. Moreover, it is the first time to show that the convergence rate, namely the upper bound of the approximation, exactly matches the lower bound of an approximation obtained by the offline/classical PCA up to a constant factor.
引用
收藏
页数：11
相关论文
共 50 条
  • [1] On the optimality of the Oja’s algorithm for online PCA
    Xin Liang
    Statistics and Computing, 2023, 33 (3)
  • [2] Oja’s Algorithm for Streaming Sparse PCA
    Department of Computer Science, University of Texas, Austin, United States
    不详
    arXiv,
  • [3] Convergence analysis of a deterministic discrete time system of Oja's PCA learning algorithm
    Yi, Z
    Ye, M
    Lv, JC
    Tan, KK
    IEEE TRANSACTIONS ON NEURAL NETWORKS, 2005, 16 (06): : 1318 - 1328
  • [4] Convergence analysis of Oja’s iteration for solving online PCA with nonzero-mean samples
    Siyun Zhou
    Yanqin Bai
    Science China Mathematics, 2021, 64 : 849 - 868
  • [5] Convergence analysis of Oja's iteration for solving online PCA with nonzero-mean samples
    Siyun Zhou
    Yanqin Bai
    Science China Mathematics, 2021, 64 (04) : 849 - 868
  • [6] Convergence analysis of Oja's iteration for solving online PCA with nonzero-mean samples
    Zhou, Siyun
    Bai, Yanqin
    SCIENCE CHINA-MATHEMATICS, 2021, 64 (04) : 849 - 868
  • [7] HETEROSKEDASTIC PCA: ALGORITHM, OPTIMALITY, AND APPLICATIONS
    Zhang, Anru R.
    Cai, T. Tony
    Wu, Yihong
    ANNALS OF STATISTICS, 2022, 50 (01): : 53 - 80
  • [8] Bootstrapping the error of Oja's algorithm
    Lunde, Robert
    Sarkar, Purnamrita
    Ward, Rachel
    ADVANCES IN NEURAL INFORMATION PROCESSING SYSTEMS 34 (NEURIPS 2021), 2021, 34
  • [9] On the equivalence of Oja's algorithm and GROUSE
    Balzano, Laura
    INTERNATIONAL CONFERENCE ON ARTIFICIAL INTELLIGENCE AND STATISTICS, VOL 151, 2022, 151
  • [10] On the equivalence of Oja's algorithm and GROUSE
    Balzano, Laura
    Proceedings of Machine Learning Research, 2022, 151 : 7014 - 7030
12345