Efficient Sparse Least Squares Support Vector Machines for Regression

To solve the sparseness problem of least squares support vector machine (LSSVM) in learning process, a training algorithm of LSSVM based on active learning is investigated. In the first stage of the algorithm, in order to solve the problem of a large number of similar training data samples, Support samples are selected by K-means clustering method. The second stage, this algorithm obtains a model using LSSVM and conducts function estimation of the all samples, calculating the error between the estimation values and the original samples, sorting support samples and selecting the best sample. Then the selected sample is added into training set to obtain new model. And the processes are repeated until the predetermined performance requirements are achieved, thus the sparse LSSVM model is obtained. The simulation on sinc function indicates that the proposed method performs more effectively than Suykens standard sparse method for removing the redundant support vector with better sparseness and robustness. The experiments on motorcycle dataset of the UCI indicate that the proposed algorithm can solve the problem of heteroscedasticity in some degree.