Detecting and quantifying ambiguity: a neural network approach

In general, it is not possible to have access to all variables that determine the behavior of a system. Once a number of measurable variables is identified, there might still exist hidden variables which influence the behavior of the system. The result is model ambiguity in the sense that, for the same (or very similar) input values, distinct outputs are obtained. In addition, the degree of ambiguity may vary across the range of input values. Therefore, to evaluate the accuracy of a model it is important to devise a method to obtain the degree of reliability for each output result. In this paper, we present such a scheme composed of two coupled neural networks, the first one computing the average predicted value and the other the reliability of the output, which is learned from the error values of the first one. As an illustration, the scheme is applied to a model for tracking slopes in a straw chamber and to a credit scoring model.