Model selection methods in multilayer perceptrons

Despite the huge amount of model selection theory for linear systems and the importance of neural networks in applied work, there is still little published work about the assessment on which model selection method works best for nonlinear systems such as Multilayer Perceptrons. Crossvalidation might be considered the most popular model selection method. It can be applied to linear as well as nonlinear learning systems, while algebraic model selection criteria are more attractive from the computational perspective, but they should take into account linear or nonlinear learning systems as well as whether regularization is used. In this paper we determine relative performance by comparing the novel algebraic criterion NNDIC, against well-known criteria for nonlinear systems such as GPE and NIC and the nonlinear ten-fold crossvalidation method (10NCV). Our results demonstrate the advantages of NNDIC in small samples scenarios for nonlinear systems which might include regularization.