Automated Test Assembly Using SAS Operations Research Software in a Medical Licensing Examination. Issue 3 (May 2020)

• Record Type: Journal Article
• Authors: Shao, Can; Liu, Silu; Yang, Hongwei; Tsai, Tsung-Hsun
• Is Part Of: Applied psychological measurement. Volume 44:Issue 3(2020)
• Page Start: 219
• Page End: 233
• Publication Date: 2020-05
• Subjects: automated test assembly -- SAS Operations Research -- item response theory; Psychometrics -- Periodicals; Psychological tests -- Periodicals; Psychology, Applied -- Periodicals; 150.1519505
• DOI: 10.1177/0146621619847169
• Languages: English
• ISSNs: 0146-6216

Abstract

Mathematical programming has been widely used by professionals in testing agencies as a tool to automatically construct equivalent test forms. This study introduces the linear programming capabilities (modeling language plus solvers) of SAS Operations Research as a platform to rigorously engineer tests on specifications in an automated manner. To that end, real items from a medical licensing test are used to demonstrate the simultaneous assembly of multiple parallel test forms under two separate linear programming scenarios: (a) constraint satisfaction (one problem) and (b) combinatorial optimization (three problems). In the four problems from the two scenarios, the forms are assembled subjected to various content and psychometric constraints. Assembled forms are next assessed using psychometric methods to ensure equivalence about all test specifications. Results from this study support SAS as a reliable and easy-to-implement platform for form assembly. Annotated codes are provided to promote further research and operational work in this area.