Evaluation of accuracy in MS lesion volumetry using realistic lesion phantoms

Rationale and Objectives. Quantitative analysis of such small structures as focal lesions in patients with multiple sclerosis (MS) is an important issue in both diagnosis and therapy monitoring. To reach clinical relevance, the reproducibility and accuracy of a proposed method have to be validated. We propose a framework for the generation of realistic digital phantoms of MS lesions of known volumes and their incorporation into a magnetic resonance (MR) data set of a healthy volunteer. Materials and Methods. We generated 54 data sets from a multispectral brain scan of a healthy volunteer with incorporated MS lesion phantoms. Lesion phantoms were created using different shapes (three), sizes (six), and orientations (three). An evaluation is carried out from a manual analysis of three human experts and two different semiautomatic approaches, with and without explicit modeling of partial volume effects (PVEs). Results. Intraobserver and interobserver studies were performed for the phantom data sets. All experts overestimated the true lesion volume for any phantom data set (median overestimation between 42.9% and 63.2%). Relative error and variability increased with decreasing lesion size. Similar results were obtained for the semiautomatic approach without PVE modeling. Only the approach with explicit PVE modeling was capable of generating accurate volumetric results with low systematic error. Conclusion. The proposed framework based on realistic lesion phantoms incorporated into an MR scan allows for quantitative assessment of the accuracy of manual and automated lesion volumetry. Results clearly show the importance of an improved gold standard in lesion volumetry beyond voxel counting.