Robust large-scale online kernel learning

The control-based approach has been proved to be effective for developing robust online learning methods. However, the existing control-based kernel methods are infeasible for large-scale modeling due to their high computational complexity. This paper aims to propose a computationally efficient control-based framework for robust large-scale kernel learning problems. By random feature approximation and robust loss function, the learning problems are first transformed into a group of linear feedback control problems with sparse discrete large-scale algebraic Riccati equations (DARE). Then, with the solutions of the DAREs, two promising algorithms are developed to address large-scale binary classification and regression problems, respectively. Thanks to the sparseness, explicit solutions rather than numerical solutions of the DAREs are derived by utilizing matrix computation techniques developed in our study. This substantially reduces the complexity, and makes the proposed algorithms computationally efficient for large-scale complex datasets. Compared with the existing benchmarks, the proposed algorithms can achieve faster convergent, more robust and accurate modeling results. Theoretical analysis and encouraging numerical results on synthetic and realistic datasets are also provided to illustrate the effectiveness and efficiency of our algorithms.