Discriminative Model Selection for Gaussian Mixture Models for Classification

The Gaussian mixture model (GMM) has been widely used in pattern recognition problems for clustering and probability density estimation. Given the number of mixture components (model order), the parameters of GMM can be estimated by the EM algorithm. The model order selection, however, remains an open problem. For classification purpose, we propose a discriminative model selection method to optimize the orders of all classes. Based on the GMMs initialized in some way, the orders of all classes are adjusted heuristically to improve the cross-validated classification accuracy. The model orders selected in this discriminative way are expected to give higher generalized accuracy than classwise model selection. Our experimental results on some UCI datasets demonstrate the superior classification performance of the proposed method.