Model Selection and Model Averaging for Mixed-Effects Models with Crossed Random Effects for Subjects and Items. (29th July 2022)

• Record Type: Journal Article
• Authors: Martínez-Huertas, José Á.; Olmos, Ricardo; Ferrer, Emilio
• Is Part Of: Multivariate behavioral research. Volume 57:Number 4(2022)
• Page Start: 603
• Page End: 619
• Publication Date: 2022-07-29
• Subjects: Mixed-effects models -- crossed random effects -- random slopes -- model selection -- model averaging -- ML -- REML; Psychometrics -- Periodicals; Psychology, Experimental -- Periodicals; Psychology, Experimental; Psychometrics; Periodicals; 150.15195
• DOI: 10.1080/00273171.2021.1889946
• Languages: English
• ISSNs: 0027-3171

Abstract

Abstract: A good deal of experimental research is characterized by the presence of random effects on subjects and items. A standard modeling approach that includes such sources of variability is the mixed-effects models (MEMs) with crossed random effects. However, under-parameterizing or over-parameterizing the random structure of MEMs bias the estimations of the Standard Errors (SEs) of fixed effects. In this simulation study, we examined two different but complementary perspectives: model selection with likelihood-ratio tests, AIC, and BIC; and model averaging with Akaike weights. Results showed that true model selection was constant across the different strategies examined (including ML and REML estimators). However, sample size and variance of random slopes were found to explain true model selection and SE bias of fixed effects. No relevant differences in SE bias were found for model selection and model averaging. Sample size and variance of random slopes interacted with the estimator to explain SE bias. Only the within-subjects effect showed significant underestimation of SEs with smaller number of items and larger item random slopes. SE bias was higher for ML than REML, but the variability of SE bias was the opposite. Such variability can be translated into high rates of unacceptable bias in many replications.