Ensemble learning for hierarchies of locally arranged models

We propose an ensemble technique to train multiple individual models for supervised learning tasks: The new method divides the input space into local regions which are modelled as a set of hyper-ellipsoids. For each local region an individual model is trained to approximate or classify data efficiently. The idea is to use locality in the input space as an useful constraint to realize diversity in an ensemble. The method automatically determines the size of the ensemble, realises an outlier detection mechanism and shows superiority over comparable methods in a benchmark test. Also, the method was extended to a hierarchical framework allowing a user to solve complex learning tasks by combining different sub-solutions and information sources.