A three-level evaluation process for segmentation methods in medical imaging

We propose an evaluation process for segmentation which is made up of three different levels. It enables us to carry out the time consuming steps only for those segmentation methods for which a successful segmentation is foreseeable. In the first level the developer of a segmentation method does a coarse analysis of the usefulness of the individual segmentation methods by means of visual assessment of the results for few image examples. Methods which have been judged useful at the first level are investigated in a second evaluation step as to the stability of the segmentation results in case of slight deviations in the images. For the reproduction of the image formation process a multitude of realizations of a given region of interest are produced by means of the bootstrap technique. At the third level of the evaluation process the segmentation methods are tested for segmentation errors. The segmentation methods are judged by means of empirical discrepancy values, and the effectiveness of a method chosen for the respective task is finally estimated.