Automated rating of patient and physician emotion in primary care visits. Issue 8 (August 2021)

• Record Type: Journal Article
• Authors: Park, Jihyun; Jindal, Abhishek; Kuo, Patty; Tanana, Michael; Lafata, Jennifer Elston; Tai-Seale, Ming; Atkins, David C.; Imel, Zac E.; Smyth, Padhraic
• Is Part Of: Patient education and counseling. Volume 104:Issue 8(2021)
• Page Start: 2098
• Page End: 2105
• Publication Date: 2021-08
• Subjects: Doctor-patient communication -- Patient-physician communication -- Doctor-patient conversation -- Machine learning -- Natural language processing -- Sentiment analysis -- Emotion classification -- Primary care visit; Patient education -- Periodicals; Health counseling -- Periodicals; Health education -- Periodicals; Counseling -- Periodicals; Patient Education -- Periodicals; Éducation des patients -- Périodiques; Counseling -- Périodiques; Éducation sanitaire -- Périodiques; 615.5071
• DOI: 10.1016/j.pec.2021.01.004
• Languages: English
• ISSNs: 0738-3991

Abstract

Highlights: The agreement of emotion ratings from the recurrent neural network model with human ratings was comparable to human-human inter-rater agreement. The neural network model performed better than the logistic regression model, which did not take into account sequence of text in the transcript. Machine learning models are able to learn meaningful representation of emotional valence information. Abstract: Objective: Train machine learning models that automatically predict emotional valence of patient and physician in primary care visits. Methods: Using transcripts from 353 primary care office visits with 350 patients and 84 physicians (Cook, 2002 [1 ], Tai-Seale et al., 2015 [2 ]), we developed two machine learning models (a recurrent neural network with a hierarchical structure and a logistic regression classifier) to recognize the emotional valence (positive, negative, neutral) (Posner et al., 2005 [3 ]) of each utterance. We examined the agreement of human-generated ratings of emotional valence with machine learning model ratings of emotion. Results: The agreement of emotion ratings from the recurrent neural network model with human ratings was comparable to that of human-human inter-rater agreement. The weighted-average of the correlation coefficients for the recurrent neural network model with human raters was 0.60, and the human rater agreement was also 0.60. Conclusions: The recurrent neural network model predicted the emotional valence of patients and physicians in primary care visits with similar reliability as human raters. Practice implications: As the first machine learning-based evaluation of emotion recognition in primary care visit conversations, our work provides valuable baselines for future applications that might help monitor patient emotional signals, supporting physicians in empathic communication, or examining the role of emotion in patient-centered care.