Comparison of receiver operating characteristic curves on the basis of optimal operating points

Rationale and Objectives. We developed a method of comparing receiver operating characteristic (ROC) curves on the basis of the utilities associated with their optimal operating points (OOPs). Methods. OOPs were computed for paired ROC curves on the basis of isocost lines in ROC space with slopes ranging from 0.1 to 3.0, For each pair of OOPs corresponding to a single isocost slope, the difference in costs and the variance of this difference was computed. A sensitivity analysis was thus obtained for the difference between the two curves over a range of isocost slopes. Three published data sets were evaluated using this technique, as well as by comparisons of areas under the curves and of true-positive fractions at fixed false-positive fractions. Results. The OOPs of paired ROC curves often occur at different false-positive fractions. Comparisons of ROC curves on the basis of OOPs may provide results that differ from comparisons of curves at a fixed false-positive fraction. Conclusion. ROC curves may be compared on the basis of utilities associated with their OOP. This comparison of the optimal performance of two diagnostic tests may differ from conventional statistical comparisons.