Focusing Narrowly on Model Fit in Factor Analysis Can Mask Construct Heterogeneity and Model Misspecification: Applied Demonstrations across Sample and Assessment Types. Issue 1 (2nd January 2023)

• Record Type: Journal Article
• Authors: Stanton, Kasey; Watts, Ashley L.; Levin-Aspenson, Holly F.; Carpenter, Ryan W.; Emery, Noah N.; Zimmerman, Mark
• Is Part Of: Journal of personality assessment. Volume 105:Issue 1(2023)
• Page Start: 1
• Page End: 13
• Publication Date: 2023-01-02
• Subjects: Personality assessment -- Periodicals; Psychology; Periodicals; 155.28
• DOI: 10.1080/00223891.2022.2047060
• Languages: English
• ISSNs: 0022-3891

Abstract

Abstract: This study builds upon research indicating that focusing narrowly on model fit when evaluating factor analytic models can lead to problematic inferences regarding the nature of item sets, as well as how models should be applied to inform measure development and validation. To advance research in this area, we present concrete examples relevant to researchers in clinical, personality, and related subfields highlighting two specific scenarios when an overreliance on model fit may be problematic. Specifically, we present data analytic examples showing that focusing narrowly on model fit may lead to (a) incorrect conclusions that heterogeneous item sets reflect narrower homogeneous constructs and (b) the retention of potentially problematic items when developing assessment measures. We use both interview data from adult outpatients ( N = 2, 149) and self-report data from adults recruited online ( N = 547) to demonstrate the importance of these issues across sample types and assessment methods. Following demonstrations with these data, we make recommendations focusing on how other model characteristics (e.g., factor loading patterns; carefully considering the content and nature of factor indicators) should be considered in addition to information provided by model fit indices when evaluating factor analytic models.