Model Reduction of Neural Network Trees Based on Dimensionality Reduction

Neural network tree (NNTree) is a hybrid model for machine learning. Compared with single model fully connected neural networks, NNTrees are more suitable for structural learning, and faster for decision making. Recently, we proposed an efficient algorithm for inducing the NNTrees based on a heuristic grouping strategy. In this paper, we try to induce smaller NNTrees based on dimensionality reduction. The goal is to induce NNTrees that are compact enough to be implemented in a VLSI chip. Two methods are investigated for dimensionality reduction. One is the principal component analysis (PCA), and another is linear discriminant analysis (LDA). We conducted experiments on several public databases, and found that the NNTree obtained after dimensionality reduction usually has less nodes and much less parameters, while the performance is comparable with the NNTree obtained without dimensionality reduction.