Machine learning algorithms to classify self-harm behaviours in New South Wales Ambulance electronic medical records: A retrospective study. (May 2022)

• Record Type: Journal Article
• Authors: Burnett, Alexander; Chen, Nicola; Zeritis, Stephanie; Ware, Sandra; McGillivray, Lauren; Shand, Fiona; Torok, Michelle
• Is Part Of: International journal of medical informatics. Volume 161(2022)
• Publication Date: 2022-05
• Subjects: Suicidal behaviour -- Epidemiology -- Machine learning -- Natural language processing -- Population surveillance; ICD-10 International Classification of Diseases 10th revision -- eMRs Electronic medical records -- NSW New South Wales -- TF-IDF Term frequency-inverse document frequency -- SVM Support Vector Machine -- MCC Matthews correlation coefficient -- PPV Positive predictive value -- NPV Negative predictive value -- PHCR Patient Health Care Record; Medical informatics -- Periodicals; Information science -- Periodicals; Computers -- Periodicals; Medical technology -- Periodicals; Medical Informatics -- Periodicals; Technology, Medical -- Periodicals; Computers; Information science; Medical informatics; Medical technology; Electronic journals; Periodicals; Electronic journals; 610.285
• DOI: 10.1016/j.ijmedinf.2022.104734
• Languages: English
• ISSNs: 1386-5056

Abstract

Highlights: Machine learning models are increasingly being used for the classification of many health problems within clinical health datasets. Improving the quality and timeliness of data on suicide and self-harm data will help to inform ongoing suicide prevention activities. Self-harm behaviour within New South Wales Ambulance eMR data can be classified using a machine learning model. The Support Vector Machine and logistic regression models performed favourably compared to manual annotation of self-harm behaviour. Abstract: Background: There is increasing interest in suicide surveillance solutions to identify non-fatal suicidal and self-harming behaviours in the Australian community not currently captured through national administrative datasets. Objective: The aim of the present study was to develop machine learning models to classify self-harm related behaviours using unstructured clinical note text from New South Wales (NSW) Ambulance data and compare their performance via traditional methods. Methods: Primary data were derived from NSW Ambulance electronic medical records (eMRs) for potential self-harm related NSW Ambulance attendances for the period 2013–2019. Data included paramedic clinical notes detailing the nature of the attendance, clinical outcome, and narrative information. We assessed sensitivity, specificity, positive predictive value, negative predictive value, F-score, and the Matthews correlation coefficient (MCC) for four algorithms (Support Vector Machine, random forest, decision tree, and logistic regression). Results: The performance of these algorithms was compared using the MCC measure. In a test sample of 3157 ambulance attendances (1349 self-harm related behaviours and 1808 unrelated), the MCC for classification of self-harm related behaviour ranged from +0.681 to +0.730. The Support Vector Machine (sensitivity = 82.7%, specificity = 89.6%, MCC = 0.730) and the logistic regression (sensitivity = 83.1%, specificity = 89.3%, MCC = 0.727) models performed best. Conclusions: This study demonstrates that machine learning models can be applied to paramedic notes within unstructured medical records to classify self-harm related behaviours. The resulting model could be used to compliment current manual abstraction of self-harm behaviours and provide more timely approximations to be used for self-harm surveillance.