Qualitative Coding in the Computational Era: A Hybrid Approach to Improve Reliability and Reduce Effort for Coding Ethnographic Interviews. (December 2021)

• Record Type: Journal Article
• Authors: Li, Zhuofan; Dohan, Daniel; Abramson, Corey M.
• Is Part Of: Socius. Volume 7(2021)
• Publication Date: 2021-12
• Subjects: computational social science -- qualitative methods -- machine learning -- natural language processing -- coding reliability -- computational ethnography; Sociology -- Research -- Periodicals; 301.07205
• DOI: 10.1177/23780231211062345
• Languages: English
• ISSNs: 2378-0231

Abstract

Sociologists have argued that there is value in incorporating computational tools into qualitative research, including using machine learning to code qualitative data. Yet standard computational approaches do not neatly align with traditional qualitative practices. The authors introduce a hybrid human-machine learning approach (HHMLA) that combines a contemporary iterative approach to qualitative coding with advanced word embedding models that allow contextual interpretation beyond what can be reliably accomplished with conventional computational approaches. The results, drawn from an analysis of 87 human-coded ethnographic interview transcripts, demonstrate that HHMLA can code data sets at a fraction of the effort of human-only strategies, saving hundreds of hours labor in even modestly sized qualitative studies, while improving coding reliability. The authors conclude that HHMLA may provide a promising model for coding data sets where human-only coding would be logistically prohibitive but conventional computational approaches would be inadequate given qualitative foci.