Subspace based additive fuzzy systems for classification and dimension reduction

In classification tasks the appearance of high dimensional feature vectors and small datasets is a common problem. It is well known that these two characteristics usually result in an oversized model with poor generalization power. In this contribution a new way to cope with such tasks is presented which is based on the assumption that in high dimensional problems almost all data points are located in a low dimensional subspace. A way is proposed to design a fuzzy system on a unified frame-work, and to use it to develop a new model for classification tasks. It is shown that the new model can be understood as an additive fuzzy system with parameter based basis functions. Different parts of the models are only defined in a subspace of the whole feature space. The subspaces are not defined a priori but are subject to an optimization procedure as all other parameters of the model. The new model has the capability to cope with high feature dimensions. The model has similarities to projection pursuit and to the mixture of experts architecture. The model is trained in a supervised manner via conjugate gradients and logistic regression, or backfitting and conjugate gradients to handle classification tasks. An efficient initialization procedure is also presented. In addition a technique based on oblique projections is presented which enlarges the capabilities of the model to use data with missing features. It is possible to use data with missing features in the training and in the classification phase. Based on the design of the model, it is possible to prune certain basis functions with a OLS (Orthogonal Least Squares) based technique in order to reduce the model size. Results are presented on an artificial and an application example.