An adaptive multiclass nearest neighbor classifier*

We consider a problem of multiclass classification, where the training sample S_n={(X-i,Y-i)}(n)(i=1) is generated from the model DOUBLE-STRUCK CAPITAL P(Y = m|X = x) = eta(m)(x), 1 <= m <= M, and eta(1)(x), horizontal ellipsis , eta(M)(x) are unknown alpha-Holder continuous functions. Given a test point X, our goal is to predict its label. A widely used k-nearest-neighbors classifier constructs estimates of eta(1)(X), horizontal ellipsis , eta(M)(X) and uses a plug-in rule for the prediction. However, it requires a proper choice of the smoothing parameter k, which may become tricky in some situations. We fix several integers n(1), horizontal ellipsis , n(K), compute corresponding n(k)-nearest-neighbor estimates for each m and each n(k) and apply an aggregation procedure. We study an algorithm, which constructs a convex combination of these estimates such that the aggregated estimate behaves approximately as well as an oracle choice. We also provide a non-asymptotic analysis of the procedure, prove its adaptation to the unknown smoothness parameter alpha and to the margin and establish rates of convergence under mild assumptions.