Assessing the coding reliability of work accidents statistical data: How coders make a difference. (December 2016)

• Record Type: Journal Article
• Authors: Jacinto, Celeste; Santos, Fernando P.; Guedes Soares, Carlos; Silva, Sílvia A.
• Is Part Of: Journal of safety research. Volume 59(2016:Dec.)
• Page Start: 9
• Page End: 21
• Publication Date: 2016-12
• Subjects: Accident data -- Content analysis -- Reliability coefficients -- Intercoder reliability -- ESAW variables; Industrial safety -- Periodicals; Accidents -- Prevention -- Periodicals; Safety -- Periodicals; Accidents, Occupational -- Periodicals; Sécurité du travail -- Périodiques; Accidents -- Prévention -- Périodiques; Accidents -- Prevention; Industrial safety; Periodicals; 363.1
• DOI: 10.1016/j.jsr.2016.09.005
• Languages: English
• ISSNs: 0022-4375

Abstract

Abstract: Introduction: This study assesses the reliability of the coding procedure for a set of variables belonging to the European Statistics of Accidents at Work (ESAW). The work focused on the Portuguese data and experience with the system. In Portugal, this task has been systematically carried out by GEP (the governmental Cabinet for Strategy and Planning), here defined as the "reference group" or "expert group." However, it is anticipated that this coding task will be performed by non-expert people, since paper-forms will be replaced by e-forms, similarly to what happened in a few EU countries. Objective: This study aims to: (a) assess the current situation, that is, to quantify reliability of data coded by GEP (reference group), and (b) assess the impact on the reliability level when the coding is carried out by non-experts (two different groups of coders). Methods: The study comprises the estimation of both intercoder and intracoder reliability for a set of 8 nominal variables. The assessment applies 3 reliability coefficients calculated by 3 software packages. Results: The results reveal that the expert group (GEP) holds good to excellent reliability (inter- and intracoder agreements), between 68–98%, while there is a considerable "loss of reliability" (− 5% to − 39%) when the coding process is transferred to other people, without special training or knowledge in this task. Conclusions: This work gives quantified evidence that reliability of coding accident data is substantially affected by the coders' profile. Moreover, certain variables, regardless of the coder, systematically hold a higher level of coding reliability than others, suggesting that certain codes may need improvement. Future studies should assess coding quality across the EU countries using the ESAW protocol. Practical applications: Directions for improving the quality of accident data and related statistics; data that is used by researchers and governmental decision-makers to derive prevention strategies. Highlights: It assesses coding reliability of 8 ESAW variables (3 different groups of coders) The "expert group" (governmental coders) has 68-98% reliability (good to excellent) Reliability loss (-5 to -39%) occurs when coded by people without training/knowledge Variable "Economic Activity of Employer" holds the highest reliability agreement Some ESAW variables must improve quality of coding, e.g.: "Mat. Agent of Deviation"