What should you optimize when building an estimation model?

When estimation models are derived from existing data, they are commonly evaluated using statistics such as mean magnitude of relative error But when the models are derived in the first place, it is usually by optimizing something else - typically, as in statistical regression, by minimizing the sum of squared deviations. How do estimation models for typical software engineering data fare, on various common accuracy statistics, if they are derived using other 'fitness functions"? In this study, estimation models are built using a variety of fitness functions, and evaluated using a wide range of accuracy statistics. We find that models based on minimizing actual errors generally out-perform models based on minimizing relative errors. Given the nature of software engineering data sets, minimizing the sum of absolute deviations seems an effective compromise.