A Comprehensive Comparison of Model Selection Methods for Testing Factorial Invariance. Issue 3 (3rd May 2020) - Liang, Xinya; Luo, Yong
Abstract : This study compared the performance of (1) model selection methods based on a point estimate (MSM-P), (2) partially Bayesian methods, and (3) fully Bayesian methods, for testing factorial invariance. The MSM-P considered included the likelihood ratio test (LRT) and a number of information criteria (IC). The Bayesian methods tested included the deviance information criterion (DIC), widely available information criterion (WAIC), and leave-one-out cross-validation (LOO). We investigated true positive and false positive rates of each model selection method under conditions varying in sample size, proportion of non-invariant items, and pattern, location and magnitude of non-invariance. The impact of various prior choices on Bayesian model selection results was also examined. Results showed that DIC, WAIC, and LOO performed comparably well, had a good balance between true and false positive rates, were flexible given the prior choices, and may complement MSM-P in factorial invariance testing. Implications and recommendations based on the findings were discussed.