Item response analysis of computational thinking practices: Test characteristics and students' learning abilities in visual programming contexts. (September 2021)

• Record Type: Journal Article
• Authors: Kong, Siu Cheung; Wang, Yi Qing
• Is Part Of: Computers in human behavior. Volume 122(2021)
• Publication Date: 2021-09
• Subjects: Computational thinking education -- Computational thinking practices -- Item response theory -- Programming education -- Test development and assessment; Interactive computer systems -- Periodicals; Man-machine systems -- Periodicals; 004.019
• DOI: 10.1016/j.chb.2021.106836
• Languages: English
• ISSNs: 0747-5632

Abstract

Abstract: Computational thinking (CT) practices have gained increasing importance in primary school education. However, there are few tools to measure primary school students' CT practices that are based on rigorous psychometric approaches. This study developed an operational definition for CT practices by exploring the key dimensions of their composition and, more importantly, developed and validated a generic test tool that can effectively assess students' CT practices. Items to measure CT practices were developed using the evidence-centered design (ECD) approach. These items prompted students to exhibit behaviors that demonstrated their skills in the four dimensions of CT practice, namely testing and debugging, algorithmic thinking, reusing and remixing, and abstracting and modularizing. A test consisting of these items was administered to 13, 956 Grade 4–6 primary school students from 56 public schools in Hong Kong. The data were analyzed based on the item response theory (IRT) approach, together with other psychometric tests. A bi-factor three-parameter logistic model was ultimately adopted for test validation, as it fitted the data better than the corresponding two-parameter and one-parameter logistic models. The results validated the four-dimensional structure of the CT practices test. The reliability, test item characteristics, and CT practice abilities of students are also discussed. Highlights: Provided a theory-driven base for the test development of computational thinking practices. Developed test tool of computational thinking practices based on the evidence-centered design approach. Testlet response analysis was adopted for test validation and refinement. The computational thinking practices test was generic for potential use in a broad range of CT contexts.