Statistical Inference for G-indices of Agreement. (August 2022) - Bonett, Douglas G.
The limitations of Cohen's κ are reviewed and an alternative G -index is recommended for assessing nominal-scale agreement. Maximum likelihood estimates, standard errors, and confidence intervals for a two-rater G -index are derived for one-group and two-group designs. A new G -index of agreement for multirater designs is proposed. Statistical inference methods for some important special cases of the multirater design also are derived. G -index meta-analysis methods are proposed and can be used to combine and compare agreement across two or more populations. Closed-form sample-size formulas to achieve desired confidence interval precision are proposed for two-rater and multirater designs. R functions are given for all results.