Inter-rater reliability of occupational exposure assessment in a case-control study of female breast cancer. Issue 10 (2nd November 2021) - Batisse, E.; Labrèche, F.; Goldberg, M. S.; Lavoué, J.; Parent, M. E.; Pasquet, R.; Richardson, L.; Siemiatycki, J.; Ho, V.
Abstract: The objective of this paper was to estimate the inter-rater reliability of expert assessments of occupational exposures. An inter-rater reliability sub-study was conducted within a population-based case-control study of postmenopausal breast cancer. Detailed information on lifetime occupational histories was obtained from participants and two industrial hygienists assigned exposures to 185 jobs using a checklist of 293 agents. Experts rated exposure for each job–agent combination according to exposure status (unexposed/exposed), confidence that the exposure occurred (possible/probable/definite), intensity (low/medium/high), and frequency (% time per week). The statistical unit of observation was each job–agent assessment (185 jobs × 293 agents = 54, 205 assessments per expert). Crude agreement, Gwet AC1/2 statistics, and Cohen's Kappa were used to estimate inter-rater agreement for confidence and intensity; for frequency, the intra-class correlation coefficient (ICC) was used. The majority of job–agent combinations were evaluated by the two experts to be not exposed (crude agreement >98% of decisions). The degree of agreement between the experts for the confidence of exposure status was Gwet AC1/2 = 0.99 (95% CI: 0.99–0.99), and for intensity, a Gwet AC2 = 0.99 (95% CI: 0.99–0.99). For frequency, an ICC of 0.31 (95% CI: 0.26–0.35) was found. A sub-analysis restricted to job–agent combinations for which the two experts agreed on exposure status revealed a moderate agreement for confidence of exposure (Gwet AC2 = 0.66) and high agreement for intensity (Gwet AC2 = 0.96). For frequency, the ICC was 0.52 (95% CI: 0.47–0.57). A high level of inter-rater agreement was found for identifying exposures and for coding intensity, but agreement was lower for the coding of frequency of exposure.