Cross Validation Can Estimate How Well Prediction Variance Correlates with Error

Cross validation for estimating the correlation between the prediction variance and the errors was proposed. The correlation between the square root of the prediction variance, s, and the absolute value of the error in prediction, was calculated. During the process of cross validation, at the points that were taken out, were computed for both errors and prediction variance. Two analytical functions, Branin-Hoo and Hartaman were widely used as benchmark problems in optimization. The approach was tested on two algebraic examples for kriging and polynomial response surface surrogates. The results suggested that the statistically based prediction variance may not always correlate well to the errors, and the surrogate with the most accurate predictions did not necessarily have the best correlation. The uncertainty structure of polynomial response surfaces was almost as good as the best kriging surrogate. Copyright © 2008 by the American Instituteof Aeronautics and Astronautics, Inc.