Estimating student ability and problem difficulty using item response theory (IRT) and TrueSkill. Issue 2 (20th May 2019)

• Record Type: Journal Article
• Authors: Lee, Youngjin
• Is Part Of: Information discovery and delivery. Volume 47:Issue 2(2019)
• Page Start: 67
• Page End: 75
• Publication Date: 2019-05-20
• Subjects: Problem solving -- User modeling -- Prediction model -- Educational data mining (EDM) -- Log file analysis -- Learning analytics (LA); Information retrieval -- Periodicals; Document delivery -- Periodicals; Digital libraries -- Periodicals; Information storage and retrieval systems -- Periodicals; 025.524
• DOI: 10.1108/IDD-08-2018-0030
• Languages: English
• ISSNs: 2398-6247

Abstract

Abstract : Purpose: The purpose of this paper is to investigate an efficient means of estimating the ability of students solving problems in the computer-based learning environment. Design/methodology/approach: Item response theory (IRT) and TrueSkill were applied to simulated and real problem solving data to estimate the ability of students solving homework problems in the massive open online course (MOOC). Based on the estimated ability, data mining models predicting whether students can correctly solve homework and quiz problems in the MOOC were developed. The predictive power of IRT- and TrueSkill-based data mining models was compared in terms of Area Under the receiver operating characteristic Curve. Findings: The correlation between students' ability estimated from IRT and TrueSkill was strong. In addition, IRT- and TrueSkill-based data mining models showed a comparable predictive power when the data included a large number of students. While IRT failed to estimate students' ability and could not predict their problem solving performance when the data included a small number of students, TrueSkill did not experience such problems. Originality/value: Estimating students' ability is critical to determine the most appropriate time for providing instructional scaffolding in the computer-based learning environment. The findings of this study suggest that TrueSkill can be an efficient means for estimating the ability of students solving problems in the computer-based learning environment regardless of the number of students.