Multilevel CFA with Ordered Categorical Data: A Simulation Study Comparing Fit Indices Across Robust Estimation Methods. Issue 1 (2nd January 2021)

• Record Type: Journal Article
• Authors: Padgett, R. Noah; Morgan, Grant B.
• Is Part Of: Structural equation modeling. Volume 28:Issue 1(2021)
• Page Start: 51
• Page End: 68
• Publication Date: 2021-01-02
• Subjects: ML-CFA -- fit-indices -- robust-estimation -- ROC; Multivariate analysis -- Periodicals; Social sciences -- Statistical methods -- Periodicals; 519.535
• DOI: 10.1080/10705511.2020.1759426
• Languages: English
• ISSNs: 1070-5511

Abstract

ABSTRACT: Within a multilevel confirmatory factor analysis framework, we investigated the ability of commonly used fit indices to discriminate between correctly specified models and misspecified models. Receiver operating characteristics (ROC) analyses were used to evaluate the performance of fit indices. Combining ROC analyses with checks of the convergence rates across Monte Carlo replications and ANOVA for investigating the variation in fit scores across replications, we found converging evidence for the utility of the investigated fit indices. Optimal thresholds based on maximizing sensitivity and specificity for detection of the true model were identified by the highest sensitivity and specificity and found to vary across different robust estimation methods (i.e., MLR, ULSMV, and WLSMV). The estimation method and sample size influenced the performance of common fit indices to detect misspecification of the level-1 model. All fit indices investigated performed poorly for detecting misspecification of the level-2 model when the level-2 sample size was below 100. We offer recommendations of commonly reported fit indices to use (and not use), cutoff criteria to use for specific estimation methods, and cautions about the use of recommended cutoff criteria for ML-CFA.