Comparing Multilevel and Classical Confirmatory Factor Analysis Parameterizations of Multirater Data: A Monte Carlo Simulation Study. Issue 1 (2nd January 2017)

• Record Type: Journal Article
• Authors: Ulitzsch, Esther; Holtmann, Jana; Schultze, Martin; Eid, Michael
• Is Part Of: Structural equation modeling. Volume 24:Issue 1(2017)
• Page Start: 80
• Page End: 103
• Publication Date: 2017-01-02
• Subjects: Monte Carlo simulation -- multilevel analysis -- multi–rater data -- multitrait–multimethod analysis -- structural equation modeling; Multivariate analysis -- Periodicals; Social sciences -- Statistical methods -- Periodicals; 519.535
• DOI: 10.1080/10705511.2016.1251846
• Languages: English
• ISSNs: 1070-5511

Abstract

Abstract : This simulation study assesses the statistical performance of two mathematically equivalent parameterizations for multitrait–multimethod data with interchangeable raters—a multilevel confirmatory factor analysis (CFA) and a classical CFA parameterization. The sample sizes of targets and raters, the factorial structure of the trait factors, and rater missingness are varied. The classical CFA approach yields a high proportion of improper solutions under conditions with small sample sizes and indicator-specific trait factors. In general, trait factor related parameters are more sensitive to bias than other types of parameters. For multilevel CFAs, there is a drastic bias in fit statistics under conditions with unidimensional trait factors on the between level, where root mean square error of approximation (RMSEA) and χ 2 distributions reveal a downward bias, whereas the between standardized root mean square residual is biased upwards. In contrast, RMSEA and χ 2 for classical CFA models are severely upwardly biased in conditions with a high number of raters and a small number of targets.