A study of re-sampling methods with regression modeling

There are an overwhelming number of applications of data mining that result in the use of regression models. For example, predicting the propensity of a customer to default on a credit card, or the likelihood that a prospect will respond to a direct marketing campaign. Unfortunately, the implementation constraints for many such useful applications restrict the type of predictive method used to simple linear or logistic regression. While the more sophisticated techniques (e.g Neural Nets [I]) have built in processes that make the resulting model the most predictive and robust, developing a robust linear/logistic regression model requires much care with an experienced hand. In business settings, most predictive models are built on a modeling data set and independently validated on a validation dataset. Often times, the modeling and validation data set have differences that cause the modeler to question whether the model will perform well in the future. This paper explores the use of resampling methods in the model building steps to help to build an optimal sample that not only fits both the modeling and validation sample well, but also holds up robustly. The resampling allows many more sample datasets to be considered and eliminates overfilling of the model sample.