Feature Bundles and their Effect on the Performance of Tree-based Evolutionary Classification and Feature Selection Algorithms

In this paper, we prove the existence of feature bundles in some classification problems. These are a set of features that while in their entirety are relevant to the target variable, any strict subset of them is completely independent of the target. Any machine learning algorithm applied to a strict subset of a feature bundle cannot produce a model that performs better than a feature-less model that always predicts the majority class. We demonstrate and discuss the effect of these feature bundles on the performance of tree-based classification learning and feature selection algorithms.