Evaluation of measures for assessing time-saving of automatic organ-at-risk segmentation in radiotherapy. (January 2020)

• Record Type: Journal Article
• Authors: Vaassen, Femke; Hazelaar, Colien; Vaniqui, Ana; Gooding, Mark; van der Heyden, Brent; Canters, Richard; van Elmpt, Wouter
• Is Part Of: Physics and imaging in radiation oncology. Volume 13(2020)
• Page Start: 1
• Page End: 6
• Publication Date: 2020-01
• Subjects: Radiotherapy -- Automatic delineation -- Contouring time -- Time-saving -- Hausdorff distance -- Dice similarity coefficient -- Surface DSC -- Added path length; Radiotherapy -- Periodicals; Radiation dosimetry -- Periodicals; Cancer -- Imaging -- Periodicals; Oncology -- Periodicals; 615.842
• DOI: 10.1016/j.phro.2019.12.001
• Languages: English
• ISSNs: 2405-6316

Abstract

Highlights: Automatic delineation software shows promising results in terms of time-saving. Standard geometry measures do not have a high correlation with delineation time. New evaluation measures were introduced: added path length (APL) and surface DSC. (Added) path length showed the highest correlation with time-recordings. This makes APL the most representative measure for clinical usefulness. Abstract: Background and purpose: In radiotherapy, automatic organ-at-risk segmentation algorithms allow faster delineation times, but clinically relevant contour evaluation remains challenging. Commonly used measures to assess automatic contours, such as volumetric Dice Similarity Coefficient (DSC) or Hausdorff distance, have shown to be good measures for geometric similarity, but do not always correlate with clinical applicability of the contours, or time needed to adjust them. This study aimed to evaluate the correlation of new and commonly used evaluation measures with time-saving during contouring. Materials and methods: Twenty lung cancer patients were used to compare user-adjustments after atlas-based and deep-learning contouring with manual contouring. The absolute time needed (s) of adjusting the auto-contour compared to manual contouring was recorded, from this relative time-saving (%) was calculated. New evaluation measures (surface DSC and added path length, APL) and conventional evaluation measures (volumetric DSC and Hausdorff distance) were correlated with time-recordings and time-savings, quantified with the Pearson correlation coefficient, R. Results: The highest correlation (R = 0.87) was found between APL and absolute adaption time. Lower correlations were found for APL with relative time-saving (R = −0.38), for surface DSC with absolute adaption time (R = −0.69) and relative time-saving (R = 0.57). Volumetric DSC and Hausdorff distance also showed lower correlation coefficients for absolute adaptation time (R = −0.32 and 0.64, respectively) and relative time-saving (R = 0.44 and −0.64, respectively). Conclusion: Surface DSC and APL are better indicators for contour adaptation time and time-saving when using auto-segmentation and provide more clinically relevant and better quantitative measures for automatically-generated contour quality, compared to commonly-used geometry-based measures.