Contouring of emerging organs-at-risk (OARS) of the female pelvis and interobserver variability: a study by the Italian association of radiotherapy and clinical oncology (AIRO)

Purpose: To provide straightforward instructions for daily practice in delineating emerging organs-at-risk (OARs) of the female pelvis and to discuss the interobserver variability in a two-step multicenter study.Methods and materials: A contouring atlas with anatomical boundaries for each emerging OAR was realized by radiation oncologists and radiologists who are experts in pelvic imaging, as per their knowledge and clinical practice. These contours were identified as quality benchmarks for the analysis subsequently carried out. Radiation oncologists not involved in setting the custom-built contouring atlas and interested in the treatment of gynecological cancer were invited to participate in this 2-step trial. In the first step all participants were supplied with a selected clinical case of locally advanced cervical cancer and had to identify emerging OARs (Levator ani muscle; Puborectalis muscle; Internal anal sphincter; External anal sphincter; Bladder base and trigone; Bladder neck; Iliac Bone Marrow; Lower Pelvis Bone Marrow; Lumbosacral Bone Marrow) based on their own personal knowledge of pelvic anatomy and experience. The suggested OARs and the contouring process were then presented at a subsequent webinar meeting with a contouring laboratory. Finally, in the second step, after the webinar meeting, each participant who had joined the study but was not involved in setting the benchmark received the custom-built contouring atlas with anatomical boundaries and was requested to delineate again the OARs using the tool provided. The Dice Similarity Coefficient (DSC) and the Jaccard Similarity Coefficient (JSC) were used to evaluate the spatial overlap accuracy of the different volume delineations and compared with the benchmark; the Hausdorff distance (HD) and the mean distance to agreement (MDA) to explore the distance between contours. All the results were reported as sample mean and standard deviation (SD). Results: Fifteen radiation oncologists from different Institutions joined the study. The participants had a high agreement degree for pelvic bones sub-structures delineation according to DICE (IBM: 0.9 +/- 0.02; LPBM: 0.91 +/- 0.01). A moderate degree according to DICE was showed for ovaries (Right: 0.61 +/- 0.16, Left: 0.72 +/- 0.05), vagina (0.575 +/- 0.13), bladder sub-structures (0.515 +/- 0.08) and EAS (0.605 +/- 0.05), whereas a low degree for the other sub-structures of the anal-rectal sphincter complex (LAM: 0.345 +/- 0.07, PRM: 0.41 +/- 0.10, and IAS: 0.4 +/- 0.07).Conclusion: This study found a moderate to low level of agreement in the delineation of the female pelvis emerging OARs, with a high degree of variability among observers. The development of delineation tools should be encouraged to improve the routine contouring of these OARs and increase the quality and consistency of radiotherapy planning.