Feature Selection Algorithm for Multiple Classifier Systems: A Hybrid Approach

Many problems in pattern classification and knowledge discovery require a selection of a subset of attributes or features to represent the patterns to be classified. The approach presented in this paper is designed mostly for multiple classifier systems with homogeneous (identical) classifiers. Such systems require many different subsets of the data set. The problem of finding the best subsets of a Given feature set is of exponential complexity. The main aim of this paper is to present ways to improve RBFS algorithm which is a feature selection algorithm. RBFS algorithm is computationally quite complex because it uses all decision-relative reducts of a given decision table. In order to increase its speed, we propose a new algorithm called ARS algorithm. The task of this algorithm is to decrease the number of the decision-relative reducts for a decision table. Experiments have shown that ARS has greatly improved the execution time of the RBFS algorithm. A small loss on the classification accuracy of the multiple classifier used on the subset created by tills algorithm has also been observed. To improve classification accuracy the simplified version of the bagging algorithm has been applied. Algorithms have been tested oil some benchmarks.