On Asymptotic Optimality of Cross-Validation Estimators for Kernel-Based Regularized System Identification

Kernel-based regularized system identification is one of the major advances in system identification in the past decade. A recent focus is to develop its asymptotic theory and it has been found that the Stein's unbiased risk estimator is asymptotically optimal (AO) in the sense of minimizing the mean squared error for prediction ability, but the empirical Bayes estimator is not AO in general. In this article, we further study the AO of various cross-validation (CV) estimators and show that the generalized CV method, leave k-out CV method, and r-fold CV method are all AO under mild assumptions, but the hold out CV method is not AO in general. We illustrate the efficacy of our theoretical results through numerical simulations.