Inference Based on the Best-Fitting Model Can Contribute to the Replication Crisis: Assessing Model Selection Uncertainty Using a Bootstrap Approach. Issue 4 (3rd July 2016)

• Record Type: Journal Article
• Authors: Lubke, Gitta H.; Campbell, Ian
• Is Part Of: Structural equation modeling. Volume 23:Issue 4(2016)
• Page Start: 479
• Page End: 490
• Publication Date: 2016-07-03
• Subjects: bootstrap procedure -- model selection uncertainty -- replication crisis; Multivariate analysis -- Periodicals; Social sciences -- Statistical methods -- Periodicals; 519.535
• DOI: 10.1080/10705511.2016.1141355
• Languages: English
• ISSNs: 1070-5511

Abstract

Abstract : Inferences and conclusions drawn from model fitting analyses are commonly based on a single "best fitting" model. If model selection and inference are carried out using the same data model selection, uncertainty is ignored. We illustrate the Type I error inflation that can result from using the same data for model selection and inference, and we then propose a simple bootstrap-based approach to quantify model selection uncertainty in terms of model selection rates. A selection rate can be interpreted as an estimate of the replication probability of a fitted model. The benefits of bootstrapping model selection uncertainty are demonstrated in growth mixture analyses of data from the National Longitudinal Study of Youth, and a 2-group measurement invariance analysis of the Holzinger–Swineford data.