Inter-rater concordance and operating definitions of radiologic nodal feature assessment in human papillomavirus-positive oropharyngeal carcinoma. (February 2022)

• Record Type: Journal Article
• Authors: Chin, Olivia; Alshafai, Laila; O'Sullivan, Brian; Su, Jie; Hope, Andrew; Bartlett, Eric; Hansen, Aaron R.; Waldron, John; Chepeha, Douglas; Xu, Wei; Huang, Shao Hui; Yu, Eugene
• Is Part Of: Oral oncology. Volume 125(2022)
• Publication Date: 2022-02
• Subjects: CT -- MRI -- Oropharyngeal carcinoma -- Human papillomavirus -- Lymph node -- Radiologic nodal feature -- Extranodal extension -- Inter-rater concordance; Mouth -- Cancer -- Periodicals; Mouth -- Tumors -- Periodicals; Mouth Diseases -- Periodicals; Mouth Neoplasms -- Periodicals; Bouche -- Cancer -- Périodiques; Bouche -- Tumeurs -- Périodiques; Tumeurs -- Périodiques; Electronic journals; 616.9943105
• DOI: 10.1016/j.oraloncology.2022.105716
• Languages: English
• ISSNs: 1368-8375

Abstract

Highlights: Definition and certainty level affects reliability of nodal feature assessment. Clearly defined nomenclature results in improved interrater reliability. Higher levels of certainty are associated with higher inter-rater agreement. Standardized radiologic taxonomy of essential nodal features are needed. Abstract: Background and purpose: This study aims to evaluate the reliability of radiologic nodal feature assessment in clinical node-positive human papillomavirus-positive oropharyngeal carcinoma. Materials and methods: Baseline CTs or MRIs of clinical node-positive human papillomavirus-positive oropharyngeal carcinoma diagnosed between 2012 and 2015 were reviewed independently by two neuroradiologists for seven nodal features: radiologic nodal involvement, cystic change, presence of necrosis, clustering, conglomeration, coalescence, and extranodal extension. Consensus operating definitions were derived after discussion. The features were re-reviewed in a randomly selected cohort. Levels of certainty (probability of presence: <25%, ∼50%, ∼75%, and >90%) were recorded. Interrater concordance was calculated using Cohen's kappa coefficient. Results: A total of 413 patients (826 necks) were eligible. At initial review, the inter-rater kappa values for: radiologic nodal involvement, cystic change, necrosis, clustering, conglomeration, coalescence, and extranodal extension were 0.92, 0.64, 0.48, 0.32, 0.32, 0.62, and 0.56, respectively. A re-review of 94 randomly selected cases (188 necks) after consolidation of operating definitions for nodal features showed that the inter-rater kappa values of these features were 0.83, 0.62, 0.58, 0.32, 0.18, 0.68, and 0.74 when considering ≥50% certainty as positive, and improved to 0.94, 0.66, 0.59, 0.33, 0.19, 0.76, and 0.86 when considering ≥75% certainty as positive. Conclusion: Clearly defined nomenclature results in improved interrater reliability when assessing radiologic nodal features, especially for coalescent adenopathy and extranodal extension. Higher levels of certainty are associated with higher inter-rater agreement. Radiology reporting should include clear definitions of clinically relevant nodal features as well as levels of certainty to serve various needs in clinical care and research.