Contribution-based approach for feature selection in linear programming-based models

Feature selection is a significant problem in building any predictive model. Linear programming models which minimize the sum of deviations reward addition of variables if the added variables can reduce the sum of deviations. Deviation occurs when a point falls on the wrong side of the discriminant surface. If the groups are not linearly separable, and if the number of features is large, it is possible to create a model where some of the features used in the model account for a very small reduction in the deviations. We propose a feature selection scheme for LP models in which we measure the effect of each variable in increasing the interclass separation.