SMALL SAMPLE ERROR RATE ESTIMATION FOR K-NN CLASSIFIERS

Small sample error rate estimators for nearest neighbor classifiers are reexamined and contrasted with the same estimators for three-nearest neighbor classifiers. The performance of the bootstrap estimators, e0 and 0.632B, is considered relative to leaving-one-out and other cross-validation estimators. Monte Carlo simulations are used to measure the performance of the error rate estimators. The experimental results are compared to previously reported simulations for both nearest neighbor classifiers and alternative classifiers. It is shown that each of the estimators has strengths and weaknesses for varying apparent and true error rate situations. A combined estimator that corrects the leaving-one-out estimator (by combining bootstrap and cross-validation estimators) gave strong results over a broad range of situations and warrants further investigation for other classifiers.