Graphical methods for classifier performance evaluation

Evaluating the performance of classifiers is not as trivial as it would seem at a first glance. Even the most widely used methods such as measuring accuracy or error rate on a test set has severe limitations. Two of the most prominent limitations of these measures are that they do not consider misclassification costs and can be misleading when the classes have very different prior probabilities. On the last years, several researches have pointed out alternative methods to evaluate the performance of learning systems. Some of those methods are based on graphical evaluation of classifiers. Usually, a graphical evaluation lets the user analyze the performance of a classifier under different scenarios, for instance, with different misclassification costs, and to select the classifier parameters setting that provides the best result. The objective of this paper is to survey some of the most used graphical methods for performance evaluation, which do not rely on precise class and cost distribution information.