A Two-Stage Decision Tree Algorithm on Constructing Hyper-Plane

How to construct the "appropriate" split hyper-plane in test nodes is the key of building decision trees. In this paper, we re-explain the process of building test nodes in terms of geometry. Based on this, we propose a method of learning the hyper-plane with two stages. The first stage searches for appropriate normal direction based on unsupervised methods (e.g. PCA, ICA etc), or supervised methods (e.g. neural network). The second stage detects the intercept of the hyper-plane in the normal direction according to some criterions, such as Gini of CART and information gain ratio (Gain Ratio) of C4.5. The experimental results conform that TSDT can improve the accuracy of univariate trees and that it is quite comparable to functional tree proposed by Gama, J. 2004, yet, needing much less learning time.