Inter-rater Agreement Indices for Multiple Informant Methodology. Issue 2 (17th February 2018)

• Record Type: Journal Article
• Authors: Lanz, Margherita; Sorgente, Angela; Tagliabue, Semira
• Is Part Of: Marriage & family review. Volume 54:Issue 2(2018)
• Page Start: 148
• Page End: 182
• Publication Date: 2018-02-17
• Subjects: family research -- inter-rater agreement -- multiple informants -- quantitative data -- research method; Family -- United States -- Periodicals; Marriage -- United States -- Periodicals; 306.8
• DOI: 10.1080/01494929.2017.1340919
• Languages: English
• ISSNs: 0149-4929

Abstract

ABSTRACT: This paper addresses the potential applications of multiple informant methodology (MIM) to family research. The MIM is used when data on the same unit of analysis are collected from more than one informant, and when the researchers' purpose requires a dyadic or group level of analysis. The MIM relies on inter-rater agreement (IRA) indices, which are needed to both estimate agreement among informants and aggregate scores from different informants. This review describes the main IRA indices: reliability within-group index ( r WG ), agreement within-group index ( a WG ), average deviation indices ( AD M and AD Md ), intra-class correlation coefficient (ICC), and random group index ( r WG ). For each index, we describe the aim, application contexts, formula(s), ranges, cutoffs, requirements to apply the index, and its possible application to family research.