Coding agricultural injury: Factors affecting coder agreement. (December 2020)

• Record Type: Journal Article
• Authors: Gorucu, Serap; Weichelt, Bryan; Redmond, Emily; Murphy, Dennis
• Is Part Of: Journal of safety research. Volume 75(2020)
• Page Start: 111
• Page End: 118
• Publication Date: 2020-12
• Subjects: Injury -- Coder agreement -- Injury source -- FAIC -- Kappa statistics; Industrial safety -- Periodicals; Accidents -- Prevention -- Periodicals; Safety -- Periodicals; Accidents, Occupational -- Periodicals; Sécurité du travail -- Périodiques; Accidents -- Prévention -- Périodiques; Accidents -- Prevention; Industrial safety; Periodicals; 363.1
• DOI: 10.1016/j.jsr.2020.08.006
• Languages: English
• ISSNs: 0022-4375

Abstract

Highlights: We calculated coder agreement level for agricultural injuries obtained from electronic news data. Coder agreement levels were determined. There was a high level of agreement between the coders. Five key themes were highlighted for coders' agreement and accurate coding. Abstract: Objectives: To determine coders' agreement level for the Occupational Injury and Illness Classification System (OIICS) source of injury and injury event codes, and the Farm and Agricultural Injury Classification (FAIC) code in the AgInjuryNews.org and to determine the effects of supplemental information and follow-up discussion in final code assignments. Methods: Two independent researchers initially coded 1304 injury cases from AgInjurynews.org using the OIICS and the FAIC coding schemes. Code agreement levels for injury source, event, and FAIC and the effect of supplemental information and follow-up discussions on final coding was assessed. Results: Coders' agreement levels were almost perfect for OIICS source and event categories at the 3-digit level, with lower agreement at the 4-digit level. By using supplemental information and follow-up discussion, coders improved the coding accuracy by an average 20% for FAIC. Supplemental information and follow-up discussions had helped finalize the disagreed codes 55% of the time for OIICS source coding assignments and 40% of time for OIICS event coding assignments for most detailed 4-digit levels. Five key themes emerged regarding accurate and consistent coding of the agricultural injuries: inclusion/exclusion based on industry classification system; inconsistent/discrepant reports; incomplete/nonspecific reports; effects of supplemental information on coding; and differing interpretations of code selection rules. Practical applications: Quantifying the level of agreement for agricultural injuries will lead to a better understanding of coding discrepancies and may uncover areas for improvement to coding scheme itself. High level of initial and final agreement with FAIC and OIICS codes suggest that these coding schemes are user-friendly and amenable to widespread use.