Prototyping: Sample Selection for Imbalanced Data

In the context of supervised learning, we are concerned with the task of identifying a subset of the example instances which maximize the predictive performance. In contrast to sampling, we do not generate new instances because we do not know how to reliably label them. We propose a simple and effective method with complexity that is linear in the size of the source data, and logarithmic in the size of the number of examples selected. We demonstrate empirically that very significant improvements are achievable on skewed data across a wide range of model types and data sets. In particular, we observe that the fraction achieving peak performance is proportional to the square root of the reciprocal of the skewness.