Statistical Inference for G-indices of Agreement. (August 2022)

• Record Type: Journal Article
• Authors: Bonett, Douglas G.
• Is Part Of: Journal of educational and behavioral statistics. Volume 47:Number 4(2022)
• Page Start: 438
• Page End: 458
• Publication Date: 2022-08
• Subjects: nominal agreement -- multirater agreement -- interval estimation -- meta-analysis -- sample-size planning; Educational statistics -- Periodicals; Social sciences -- Statistical methods -- Periodicals; 370.2
• DOI: 10.3102/10769986221088561
• Languages: English
• ISSNs: 1076-9986

Abstract

The limitations of Cohen's κ are reviewed and an alternative G -index is recommended for assessing nominal-scale agreement. Maximum likelihood estimates, standard errors, and confidence intervals for a two-rater G -index are derived for one-group and two-group designs. A new G -index of agreement for multirater designs is proposed. Statistical inference methods for some important special cases of the multirater design also are derived. G -index meta-analysis methods are proposed and can be used to combine and compare agreement across two or more populations. Closed-form sample-size formulas to achieve desired confidence interval precision are proposed for two-rater and multirater designs. R functions are given for all results.