Development of a keyword library for capturing PRO-CTCAE-focused "symptom talk" in oncology conversations. Issue 1 (9th February 2023)

• Record Type: Journal Article
• Authors: Durieux, Brigitte N; Zverev, Samuel R; Tarbi, Elise C; Kwok, Anne; Sciacca, Kate; Pollak, Kathryn I; Tulsky, James A; Lindvall, Charlotta
• Is Part Of: JAMIA open. Volume 6:Issue 1(2023)
• Publication Date: 2023-02-09
• Subjects: natural language processing -- patient-reported outcome measures -- computing methodologies -- signs and symptoms -- symptom assessment; Medical informatics -- Periodicals; 610.285
• DOI: 10.1093/jamiaopen/ooad009
• Languages: English
• ISSNs: 2574-2531

Abstract

Abstract: Objectives: As computational methods for detecting symptoms can help us better attend to patient suffering, the objectives of this study were to develop and evaluate the performance of a natural language processing keyword library for detecting symptom talk, and to describe symptom communication within our dataset to generate insights for future model building. Materials and Methods: This was a secondary analysis of 121 transcribed outpatient oncology conversations from the Communication in Oncologist-Patient Encounters trial. Through an iterative process of identifying symptom expressions via inductive and deductive techniques, we generated a library of keywords relevant to the Patient-Reported Outcome version of the Common Terminology Criteria for Adverse Events (PRO-CTCAE) framework from 90 conversations, and tested the library on 31 additional transcripts. To contextualize symptom expressions and the nature of misclassifications, we qualitatively analyzed 450 mislabeled and properly labeled symptom-positive turns. Results: The final library, comprising 1320 terms, identified symptom talk among conversation turns with an F1 of 0.82 against a PRO-CTCAE-focused gold standard, and an F1 of 0.61 against a broad gold standard. Qualitative observations suggest that physical symptoms are more easily detected than psychological symptoms (eg, anxiety), and ambiguity persists throughout symptom communication. Discussion: This rudimentary keyword library captures most PRO-CTCAE-focused symptom talk, but the ambiguity of symptom speech limits the utility of rule-based methods alone, and limits to generalizability must be considered. Conclusion: Our findings highlight opportunities for more advanced computational models to detect symptom expressions from transcribed clinical conversations. Future improvements in speech-to-text could enable real-time detection at scale. Lay Summary: This article describes the development of a keyword library for detecting symptoms, based on the way patients and clinicians verbally express symptoms in conversation. Keyword libraries are rudimentary natural language processing tools that can search text for language related to a predefined concept; as research emphasizes patient-reported outcomes in the serious illness setting, we sought to detect the 80 symptoms which have been established as relevant to patient-reported outcomes. This library, based on language from conversation transcripts, was supplemented with symptom expressions from prior literature and evaluated for its ability to detect symptom content in a given speaker turn. The library was able to detect most speaker turns containing patient reported outcome-relevant symptoms; qualitative exploration revealed that physical symptoms were most easily detected, and established ambiguity as a pressing issue in detecting symptoms via computational methods. This library provides a tool for aiding symptom detection, but its rudimentary status and the limitations of rule-based methods and language generalizability limit its use alone. More advanced natural language processing methods may enable scalable real-time symptom detection from emerging conversation data such as that accessible via telehealth. As these tools are developed, it will remain important to capture the way patients express symptoms.