An Improved Nadaraya-Watson Kernel Regression Method Based on Cross Validation for Small Samples

Regression problem is to find the appropriate relationship between explanatory and response variable(s). Current regression algorithms generally require sufficient samples, while insufficient samples may cause unsatisfactory predictions. To solve this problem, a regression algorithm based on Nadaraya-Watson kernel regression (NWKR), is proposed. In the proposed framework, the original NWKR algorithm is improved by parameter optimization with Cross-Validation (CV). The Leave-one-out Cross-Validation (LOO-CV) error is taken as the objective function, and the bandwidth parameter which is critical in NWKR is optimized by minimum LOO-CV error. Three loss functions are considered in the objective function to compare performance. Some experiments is taken to compare prediction performance. The results show that the e-insensitive loss function is outperform than other two loss functions, and the proposed improved NWKR has a higher prediction accuracy than some regression algorithms in small training sample.