Measurement and structural invariance of cognitive ability tests after computer-based training. (April 2019)

• Record Type: Journal Article
• Authors: Hermes, Michael; Albers, Frank; Böhnke, Jan R.; Huelmann, Gerrit; Maier, Julia; Stelling, Dirk
• Is Part Of: Computers in human behavior. Volume 93(2019)
• Page Start: 370
• Page End: 378
• Publication Date: 2019-04
• Subjects: Computer-based training -- Computer-based testing -- Cognitive ability -- Measurement invariance -- Test fairness; Interactive computer systems -- Periodicals; Man-machine systems -- Periodicals; 004.019
• DOI: 10.1016/j.chb.2018.11.040
• Languages: English
• ISSNs: 0747-5632

Abstract

Abstract: Ability tests are core elements in performance research as well as in applied contexts and are increasingly carried out using computer-based versions. In the last few decades a whole training and coaching industry has developed to prepare individuals for computer-based assessments. Evidence suggests that such commercial training programs can result in score gains in ability tests, thereby creating an advantage for those who can afford it and challenging the fairness of ability assessment. As a consequence, several authors recommended freely offering training software to all participants to increase measurement fairness. However, it is still an open question whether the unsupervised use of training software could have an impact on the measurement properties of ability tests. The goal of the present study is to fill this gap by examining the subjects' ability scores for measurement and structural invariance across different amounts of computer-based training. Structural equation modeling was employed in a sample of 15, 752 applicants who participated in high-stakes assessments with computer-based ability tests. Across different training amounts, our analyses supported measurement and structural invariance of ability scores. In conclusion, free training software is a means that provides fair preparation opportunities without changing the measurement properties of the tests. Highlights: The link between computer-based training and high-stakes assessments was investigated. Training and selection test data of 15, 752 pilot trainee applicants was analyzed. The amount of training predicted test performance in curvilinear fashion as expected. The ability test scores' structure was invariant across different amounts of training. Free training was not linked to the psychometric structure of the high-stakes tests.