A Comparison of Procedures for Estimating Person Reliability Parameters in the Graded Response Model. Issue 3 (2nd September 2018)

• Record Type: Journal Article
• Authors: LaHuis, David M.; Bryant‐Lees, Kinsey B.; Hakoyama, Shotaro; Barnes, Tyler; Wiemann, Andrea
• Is Part Of: Journal of educational measurement. Volume 55:Issue 3(2018)
• Page Start: 421
• Page End: 432
• Publication Date: 2018-09-02
• Subjects: Educational tests and measurements -- Periodicals; 371.2605
• DOI: 10.1111/jedm.12186
• Languages: English
• ISSNs: 0022-0655

Abstract

Abstract: Person reliability parameters (PRPs) model temporary changes in individuals' attribute level perceptions when responding to self‐report items (higher levels of PRPs represent less fluctuation). PRPs could be useful in measuring careless responding and traitedness. However, it is unclear how well current procedures for estimating PRPs can recover parameter estimates. This study assesses these procedures in terms of mean error (ME), average absolute difference (AAD), and reliability using simulated data with known values. Several prior distributions for PRPs were compared across a number of conditions. Overall, our results revealed little differences between using the χ or lognormal distributions as priors for estimated PRPs. Both distributions produced estimates with reasonable levels of ME; however, the AAD of the estimates was high. AAD did improve slightly as the number of items increased, suggesting that increasing the number of items would ameliorate this problem. Similarly, a larger number of items were necessary to produce reasonable levels of reliability. Based on our results, several conclusions are drawn and implications for future research are discussed.