Integrating incremental feature weighting into Naive Bayes text classifier

In the real-world operational environment, text classification systems should handle the problem of incomplete training set and no prior knowledge of feature space. In this regard, the most appropriate algorithm for operational text classification is the Naive Bayes since it is easy to incrementally update its pre-learned classification model and feature space. Our work mainly focuses on improving Naive Bayes classifier through feature weighting strategy. The basic idea is that parameter estimation of Naive Bayes can consider the degree of feature importance as well as feature distribution. In addition, we have extended a conventional algorithm for incremental feature update for developing a dynamic feature space in operational environment. Through experiments using the Reuters-21578 and the 20Newsgroup benchmark collections, we show that the traditional multinomial Naive Bayes classifier can be significantly improved by X-2-statistic based feature weighting.