A hybrid naive Bayes approach for information filtering

Naive Bayes has been widely used in the field of machine learning research for many years. While it is fast and easy to implement, its performance in comparison to other machine learning methods is not ideal, In this paper, we present a hybrid approach using naive Bayes for information filtering. This approach differs from previous approaches in that it uses Multivariate Bernoulli Model and Multinomial Model successively. We report on the performance of our proposed approach using Reuters-21578 and 20 Newsgroups data. In the filtering process, we first use Multivatiate Bernoulli Model to estimate the preexamined probability for words appear in a document. Subsequently, the Multinomial Model is used to estimate the post-examined probability for final classification. We show that with sufficient training data, this hybrid approach can achieve higher F-measure score than using Multivariate Bernoulli Model or Multinomial Model alone. It can even achieve competitive results as compared to the highly complex learning method such as Support Vector Machine (SVM) with less computational time.