An Examination of Different Methods of Setting Cutoff Values in Person Fit Research. Issue 1 (2nd January 2019)

• Record Type: Journal Article
• Authors: Mousavi, Amin; Cui, Ying; Rogers, Todd
• Is Part Of: International journal of testing. Volume 19:Issue 1(2019)
• Page Start: 1
• Page End: 22
• Publication Date: 2019-01-02
• Subjects: person fit -- cutoff value -- monte carlo -- item response theory -- large scale assessment; Psychological tests -- Periodicals; Educational tests and measurements -- Periodicals; 150.28705
• DOI: 10.1080/15305058.2018.1464010
• Languages: English
• ISSNs: 1530-5058

Abstract

Abstract : This simulation study evaluates four different methods of setting cutoff values for person fit assessment, including (a) using fixed cutoff values either from theoretical distributions of person fit statistics, or arbitrarily chosen by the researchers in the literature; (b) using the specific percentile rank of empirical sampling distribution of person fit statistics from simulated fitting responses; (c) using bootstrap method to estimate cutoff values of empirical sampling distribution of person fit statistics from simulated fitting responses; and (d) using the p-value methods to identify misfitting responses conditional on ability levels. The Snijders'l z * (2001 ), as an index with known theoretical distribution, van der Flier's U3 (1982 ) and Sijtsma's H T coefficient (1986), as indices with unknown theoretical distribution, were chosen. According to the simulation results, different methods of setting cutoff values tend to produce different levels of Type I error and detection rates, indicating it is critical to select an appropriate method for setting cutoff values in person fit research.