Bayes Theorem and Information Gain Based Feature Selection for Maximizing the Performance of Classifiers

Features play a very important role in the task of pattern classification. Consequently, the selection of suitable features is necessary as most of the raw data might be redundant or irrelevant to the recognition of patterns. In some cases, the classifier can not perform well because of the large number of redundant features. This paper presents a novel evolving feature selection algorithms taking the advantages of Bayes Theorem and Information Gain to improve the predictive accuracy. Bayes theorem is used to discover dependency information among features. In addition to that, feature selection has been improvised by Information Gain which selects features based on their importance. Different features play different roles in classifying datasets. Unwanted features will result in error information during classification which will reduce classification precision. The proposed feature selection can remove these distractions to improve classification performance. As shown in the experimental results, after feature selection using the Bayes theorem and Information gain to control false discovery rate, the classification performance of DT's and NB classifiers were significantly improved.