Mutual Information Using Sample Variance for Text Feature Selection

Feature selection improves the training speed of the classifier without affecting its predictive capability. It selects a subset of most informative words (terms) from the set of all words. Term distribution affects the feature selection process, e.g. an even distribution of terms in a specific class ensures a higher association of these terms with that class, but an even distribution in almost classes shows a lesser association. This paper computes sample variance using standard Mutual Information (MI) method to measure the variations in distribution of terms. MI method assigns a higher rank to the terms distributed in a specific category (i.e. rare terms) which shows it strong influence with the rare terms than common terms (i.e. terms which most frequently in almost classes). To address this issue, a new text feature selection method named Mutual Information Using Sample Variance (MIUSV) is proposed in this paper. It considers sample variance in term distribution while computing the Mutual Information score of the term. Multinomial Naive Bayes (MNB) and k Nearest Neighbor (kNN) classifiers model, check the utilities of the selected terms by the proposed MIUSV. These models classify four standard text data sets, viz. Webkb, 20Newsgroup, Ohsumed10, and Ohsumed23. Two standard performance measures named Macro-F1 and Micro-F1 show a significant improvement in the results using proposed MIUSV method.