Generalisation bounds for kernel PCA through PAC-Bayes learning

Principal component analysis(PCA) is a popular method for dimension reduction and has attracted an unfailing interest for decades. More recently, kernel PCA (KPCA) has emerged as an extension of PCA, but despite its use in practice, a sound theoretical understanding of KPCA is missing. We contribute several empirical generalisation bounds on the efficiency of KPCA, involving the empirical eigenvalues of the kernel Gram matrix. Our bounds are derived through the use of probably approximately correct (PAC)-Bayes theory and highlight the importance of some desirable properties of datasets, expressed as variance-typed terms, to attain fast rates, achievable for a wide class of kernels.