Non-parametric predictive inference for the validation of credit rating systems

Credit rating or credit scoring systems are important tools for estimating the obligor's creditworthiness and for providing an indication of the obligor's future status. The discriminatory power of a credit rating or credit scoring system refers to its ex ante ability to distinguish between two or more classes of borrowers. One of the most popular tools for the validation of the power of credit rating or credit scoring models to distinguish between two (or more) classes of borrowers is the receiver operating characteristic (ROC) curve (hypersurface) and its widely used overall summary, the area (hypervolume) under the curve (hypersurface). As the end goal of building such models is to predict and quantify uncertainty about future loans, prediction methods are especially valuable in this context. For this, non-parametric predictive inference is a promising candidate for such inference as it is a frequentist statistical method that is explicitly aimed at using few modelling assumptions, enabled through the use of lower and upper probabilities to quantify uncertainty. The aim of the paper is to introduce non-parametric predictive inference for ROC analysis within a banking context, for which novel results on ROC hypersurfaces for more than three groups are presented. Examples are provided to illustrate the method.