A Comprehensive Approach for Assessing Person Fit With Test–Retest Data. (August 2014)

• Record Type: Journal Article
• Authors: Ferrando, Pere J.
• Is Part Of: Educational and psychological measurement. Volume 74:Number 4(2014:Aug.)
• Page Start: 585
• Page End: 610
• Publication Date: 2014-08
• Subjects: person-fit -- test–retest data -- stability designs -- measurement of change -- retest effects; Educational tests and measurements -- Periodicals; Psychological tests -- Periodicals; 151.205
• DOI: 10.1177/0013164413518555
• Languages: English
• ISSNs: 0013-1644

Abstract

Item response theory (IRT) models allow model–data fit to be assessed at the individual level by using person-fit indices. This assessment is also feasible when IRT is used to model test–retest data. However, person-fit developments for this type of modeling are virtually nonexistent. This article proposes a general person-fit approach for test–retest data, which is based on practical likelihood-based indices. The approach is intended for two types of assumption regarding trait levels—stability and change—and can be used with a variety of IRT models. It consists of two groups of indices: (a) overall indices based on the full test–retest pattern, which are more powerful and are intended to flag a respondent as potentially inconsistent; and (b) partial indices intended to provide additional information about the location and sources of misfit. Furthermore, because the overall procedures assume local independence under repetition, a statistic for assessing the presence of retest effects at the individual level is also proposed. The functioning of the procedures was assessed by using simulation and is illustrated with two empirical studies: a stability study based on graded-response items and a change study based on binary items. Finally, limitations and further lines of research are discussed.