FAST AND ACCURATE GAUSSIAN KERNEL RIDGE REGRESSION USING MATRIX DECOMPOSITIONS FOR PRECONDITIONING

被引:2
|
作者
Shabat, Gil [1 ]
Choshen, Era [1 ]
Ben Or, Dvir [1 ]
Carmel, Nadav [1 ]
机构
[1] Playtika AI Res Lab, Herzliyya, Israel
关键词
kernel ridge regression; CUR; interpolative decomposition; preconditioner; randomized matrix decompositions; RANDOMIZED ALGORITHMS; EFFICIENT ALGORITHMS; NYSTROM METHOD; RANK; COMPRESSION; CUR;
D O I
10.1137/20M1343993
中图分类号
O29 [应用数学];
学科分类号
070104 ;
摘要
This paper presents a preconditioner-based method for solving a kernel ridge regression problem. In contrast to other methods, which utilize either fast matrix-vector multiplication or a preconditioner, the suggested approach uses randomized matrix decompositions for building a preconditioner with a special structure that can also utilize fast matrix-vector multiplications. This hybrid approach is efficient in reducing the condition number, exact, and computationally efficient, enabling the processing of large datasets with computational complexity linear to the number of data points. Also, a theoretical upper bound for the condition number is provided. For Gaussian kernels, we show that given a desired condition number, the rank of the needed preconditioner can be determined directly from the dataset.
引用
收藏
页码:1073 / 1095
页数:23
相关论文
共 50 条
  • [1] FASTER KERNEL RIDGE REGRESSION USING SKETCHING AND PRECONDITIONING
    Avron, Haim
    Clarkson, Kenneth L.
    Woodruff, David P.
    [J]. SIAM JOURNAL ON MATRIX ANALYSIS AND APPLICATIONS, 2017, 38 (04) : 1116 - 1138
  • [2] Accurate, Fast and Scalable Kernel Ridge Regression on Parallel and Distributed Systems
    You, Yang
    Demmel, James
    Hsieh, Cho-Jui
    Vuduc, Richard
    [J]. INTERNATIONAL CONFERENCE ON SUPERCOMPUTING (ICS 2018), 2018, : 307 - 317
  • [3] Accurate Prediction of Gas Compressibility Factor using Kernel Ridge Regression
    Maalouf, Maher
    Khoury, Naji
    Homouz, Dirar
    Polychronopoulou, Kyriaki
    [J]. 2019 FOURTH INTERNATIONAL CONFERENCE ON ADVANCES IN COMPUTATIONAL TOOLS FOR ENGINEERING APPLICATIONS (ACTEA), 2019,
  • [4] Gaussian process regression: Optimality, robustness, and relationship with kernel ridge regression
    Wang, Wenjia
    Jing, Bing-Yi
    [J]. JOURNAL OF MACHINE LEARNING RESEARCH, 2022, 23 : 1 - 67
  • [5] Gaussian process regression: Optimality, robustness, and relationship with kernel ridge regression
    Wang, Wenjia
    Jing, Bing-Yi
    [J]. Journal of Machine Learning Research, 2022, 23
  • [6] Sketch Kernel Ridge Regression Using Circulant Matrix: Algorithm and Theory
    Yin, Rong
    Liu, Yong
    Wang, Weiping
    Meng, Dan
    [J]. IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS, 2020, 31 (09) : 3512 - 3524
  • [7] Fast & Accurate Gaussian Kernel Density Estimation
    Heer, Jeffrey
    [J]. 2021 IEEE VISUALIZATION CONFERENCE - SHORT PAPERS (VIS 2021), 2021, : 11 - 15
  • [8] Fast Randomized Kernel Ridge Regression with Statistical Guarantees
    El Alaoui, Ahmed
    Mahoney, Michael W.
    [J]. ADVANCES IN NEURAL INFORMATION PROCESSING SYSTEMS 28 (NIPS 2015), 2015, 28
  • [9] Fast Statistical Leverage Score Approximation in Kernel Ridge Regression
    Chen, Yifan
    Yang, Yun
    [J]. 24TH INTERNATIONAL CONFERENCE ON ARTIFICIAL INTELLIGENCE AND STATISTICS (AISTATS), 2021, 130
  • [10] Face recognition using kernel ridge regression
    An, Senjian
    Liu, Wanquan
    Venkatesh, Svetha
    [J]. 2007 IEEE CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION, VOLS 1-8, 2007, : 1033 - +