Assessing privacy risks in population health publications using a checklist-based approach. (10th November 2017)

• Record Type: Journal Article
• Authors: O'Keefe, Christine M; Ickowicz, Adrien; Churches, Tim; Westcott, Mark; O'Sullivan, Maree; Khan, Atikur
• Is Part Of: Journal of the American Medical Informatics Association. Volume 25:Number 3(2018)
• Page Start: 315
• Page End: 320
• Publication Date: 2017-11-10
• Subjects: data anonymization -- confidentiality -- privacy -- biomedical research -- health services research; Medical informatics -- Periodicals; Information Services -- Periodicals; Medical Informatics -- Periodicals; Médecine -- Informatique -- Périodiques; Informatica; Geneeskunde; Informatique médicale; Computer network resources; Electronic journals; 610.285
• DOI: 10.1093/jamia/ocx129
• Languages: English
• ISSNs: 1067-5027

Abstract

Abstract: Objective: Recent growth in the number of population health researchers accessing detailed datasets, either on their own computers or through virtual data centers, has the potential to increase privacy risks. In response, a checklist for identifying and reducing privacy risks in population health analysis outputs has been proposed for use by researchers themselves. In this study we explore the usability and reliability of such an approach by investigating whether different users identify the same privacy risks on applying the checklist to a sample of publications. Methods: The checklist was applied to a sample of 100 academic population health publications distributed among 5 readers. Cohen's κ was used to measure interrater agreement. Results: Of the 566 instances of statistical output types found in the 100 publications, the most frequently occurring were counts, summary statistics, plots, and model outputs. Application of the checklist identified 128 outputs (22.6%) with potential privacy concerns. Most of these were associated with the reporting of small counts. Among these identified outputs, the readers found no substantial actual privacy concerns when context was taken into account. Interrater agreement for identifying potential privacy concerns was generally good. Conclusion: This study has demonstrated that a checklist can be a reliable tool to assist researchers with anonymizing analysis outputs in population health research. This further suggests that such an approach may have the potential to be developed into a broadly applicable standard providing consistent confidentiality protection across multiple analyses of the same data.