Digital ITEMS Module 1: Reliability in Classical Test Theory. Issue 2 (26th June 2018)

• Record Type: Journal Article
• Authors: Lewis, Charlie; Chajewski, Michael; Rupp, André A.
• Is Part Of: Educational measurement. Volume 37:Issue 2(2018)
• Page Start: 71
• Page End: 72
• Publication Date: 2018-06-26
• Subjects: classical test theory -- congeneric model -- Cronbach's α -- KR‐20 -- KR‐21 -- parallel model -- Pearson correlation -- reliability -- Rulon formula -- Spearman–Brown formula -- tau‐equivalent model; Educational tests and measurements -- Periodicals; Educational tests and measurements -- United States -- Periodicals; 371.262
• DOI: 10.1111/emip.12209
• Languages: English
• ISSNs: 0731-1745

Abstract

Abstract: In this ITEMS module, we provide a two‐part introduction to the topic of reliability from the perspective of classical test theory (CTT). In the first part, which is directed primarily at beginning learners, we review and build on the content presented in the original didactic ITEMS article by Traub and Rowley (1991). Specifically, we discuss the notion of reliability as an intuitive everyday concept to lay the foundation for its formalization as a reliability coefficient via the basic CTT model. We then walk through the step‐by‐step computation of key reliability indices and discuss the data collection conditions under which each is most suitable. In the second part, which is directed primarily at intermediary learners, we present a distribution‐centered perspective on the same content. We discuss the associated assumptions of various CTT models ranging from parallel to congeneric, and review how these affect the choice of reliability statistics. Throughout the module, we use a customized Excel workbook with sample data and basic data manipulation functionalities to illustrate the computation of individual statistics and to allow for structured independent exploration. In addition, we provide quiz questions with diagnostic feedback as well as short videos that walk through sample exercises within the workbook.