Approximate kernel partial least squares

As an extension of partial least squares (PLS), kernel partial least squares (KPLS) is an very important methods to find nonlinear patterns from data. However, the application of KPLS to large-scale problems remains a big challenge, due to storage and computation issues in the number of examples. To address this limitation, we utilize randomness to design scalable new variants of the kernel matrix to solve KPLS. Specifically, we consider the spectral properties of low-rank kernel matrices constructed as sums of random feature dot-products and present a new method called randomized kernel partial least squares (RKPLS) to approximate KPLS. RKPLS can alleviate the computation requirements of approximate KPLS with linear space and computation in the sample size. Theoretical analysis and experimental results show that the solution of our algorithm converges to exact kernel matrix in expectation.