On the optimality of the Oja's algorithm for online PCA

被引：3
|
作者：
Liang, Xin [1 ,2 ]
机构：
[1] Tsinghua Univ, Yau Math Sci Ctr, Beijing 100084, Peoples R China
[2] Yanqi Lake Beijing Inst Math Sci & Applicat, Beijing 101408, Peoples R China
来源：
STATISTICS AND COMPUTING | 2023年 / 33卷 / 03期
基金：
中国国家自然科学基金;
关键词：
Principal component analysis; Stochastic approximation; High-dimensional data; Oja's algorithm; STOCHASTIC-APPROXIMATION; PRINCIPAL;
D O I：
10.1007/s11222-023-10229-z
中图分类号：
TP301 [理论、方法];
学科分类号：
081202 ;
摘要：
In this paper we analyze the behavior of the Oja's algorithm for online/streaming principal component subspace estimation. It is proved that with high probability it performs an efficient, gap-free, global convergence rate to approximate an principal component subspace for any sub-Gaussian distribution. Moreover, it is the first time to show that the convergence rate, namely the upper bound of the approximation, exactly matches the lower bound of an approximation obtained by the offline/classical PCA up to a constant factor.
引用
收藏
页数：11
相关论文
共 50 条
  • [41] Online PCA with Optimal Regrets
    Nie, Jiazhong
    Kotlowski, Wojciech
    Warmuth, Manfred K.
    ALGORITHMIC LEARNING THEORY (ALT 2013), 2013, 8139 : 98 - 112
  • [42] Online PCA with Optimal Regret
    Nie, Jiazhong
    Kotlowski, Wojciech
    Warmuth, Manfred K.
    JOURNAL OF MACHINE LEARNING RESEARCH, 2016, 17 : 1 - 49
  • [43] Online Compressed Robust PCA
    Pan, Pingbo
    Feng, Jiashi
    Chen, Ling
    Yang, Yi
    2017 INTERNATIONAL JOINT CONFERENCE ON NEURAL NETWORKS (IJCNN), 2017, : 1041 - 1048
  • [44] A roundoff error analysis of the Oja's subspace rule
    Szabo, T
    Horvath, G
    1997 IEEE INTERNATIONAL CONFERENCE ON ACOUSTICS, SPEECH, AND SIGNAL PROCESSING, VOLS I - V: VOL I: PLENARY, EXPERT SUMMARIES, SPECIAL, AUDIO, UNDERWATER ACOUSTICS, VLSI; VOL II: SPEECH PROCESSING; VOL III: SPEECH PROCESSING, DIGITAL SIGNAL PROCESSING; VOL IV: MULTIDIMENSIONAL SIGNAL PROCESSING, NEURAL NETWORKS - VOL V: STATISTICAL SIGNAL AND ARRAY PROCESSING, APPLICATIONS, 1997, : 3297 - 3300
  • [45] Using PCA and Pareto optimality to select flexible manufacturing systems
    Rezaie, K.
    Haeri, Abdorrahman
    2011 IEEE INTERNATIONAL SYSTEMS CONFERENCE (SYSCON 2011), 2011, : 538 - 541
  • [46] A Geometric Newton Method for Oja's Vector Field
    Absil, P. -A.
    Ishteva, M.
    De Lathauwer, L.
    Van Huffel, S.
    NEURAL COMPUTATION, 2009, 21 (05) : 1415 - 1433
  • [47] Statistical Optimality and Computational Efficiency of Nyström Kernel PCA
    Sterge, Nicholas
    Sriperumbudur, Bharath K.
    Journal of Machine Learning Research, 2022, 23
  • [48] On the Optimality of Clickthrough Fees in Online Markets
    Baye, Michael R.
    Gao, Xiaxun
    Morgan, John
    ECONOMIC JOURNAL, 2011, 121 (556): : F340 - F367
  • [49] On the Optimality of Gauss's Algorithm over Euclidean Imaginary Quadratic Fields
    Porter, Christian
    Lyu, Shanxiang
    Ling, Cong
    2019 IEEE INFORMATION THEORY WORKSHOP (ITW), 2019, : 185 - 189
  • [50] ROIPCA: an online memory-restricted PCA algorithm based on rank-one updates
    Mitz, Roy
    Shkolnisky, Yoel
    INFORMATION AND INFERENCE-A JOURNAL OF THE IMA, 2023, 12 (04)
12345