The Characteristics of Kernel and Kernel-based Learning

In the application of Support Vector Machines (SVM), if the data points are not linearly separable in the original space, it is desirable to find a mapping function to map the data into the high-dimensional space, and then classify it. However, the mapped target space is often very high-dimensional or even infinite-dimensional, so it is necessary to find a function instead of the operation of finding the inner product of the vector in the high-dimensional space. Thus, this function is named as kernel function. The selection of the kernel function is required to satisfy Mercer's theorem, that is, the arbitrary Gram matrix of the kernel function in the sample space is a semi-positive definite matrix. Furthermore, kernel method is also an approach to achieve efficient calculation. It can make use of kernel function to carry out synchronous computation of nonlinear mapping in linear learning machine, so that the computational complexity is independent of the dimension of the high-dimensional feature space. Kernel function subtly solves the above problem. In high-dimension, the inner product of the vector can he calculated by the kernel function of the low-dimensional point. This technique is called kernel trick. The advantage of kernel trick is that it does not need to explicitly define the feature space and mapping function, but only need to select a suitable kernel function. In this paper, we first introduce the basic definitions of kernel function and RKHS. On this basis, the least squares learning problem of Gaussian kernel is studied. Finally, the influence of the selection of Gaussian kernel function parameters on the learning algorithm is verified by computer simulation.