Simplification of majority-voting classifiers using binary decision diagrams

Various versions of the majority-voting classification method have been proposed in recent years as a strategy for improving classification performance. This method generates multiple decision trees from training examples and performs majority voting of classification results from these decision trees in order to classify test examples. In this method, however, since the target concept is represented in multiple decision trees, its readability is poor. This property makes it ineffective in knowledge-base construction. To enable the majority-voting classification method to be applied to knowledge-base construction, this paper proposes a simplification method that converts the entire majority-voting classifier into compact disjunctive normal form (DNF) formulas. A significant feature of this method is the use of binary decision diagrams (BDDs) as internal expressions in the conversion process to achieve high-speed simplification. A problem that must be addressed here is the BDD input variable ordering scheme. This paper proposes an ordering scheme based on the order of variables in the decision trees. The simplification method has been applied to several real-world data sets of the Irvine Database and to data from medical diagnosis domain. It was found that the description size of the majority-voting classifier after simplification was on the average from 1.2 to 2.7 times that of a single decision tree and was less than one-third the size of a majority-voting classifier before simplification. Therefore, the method is effective in reducing the description size and should be applicable to the knowledge acquisition process. Using the input variable ordering scheme proposed here, high-speed simplification of several seconds to several tens of seconds is achieved on a Sun SPARC-server 10 workstation.