Computer‐based evaluation to assess students' learning for the multiple‐choice question–based exams: CBE‐MCQs software tool. Issue 6 (2nd August 2020)

• Record Type: Journal Article
• Authors: Solak, Serdar; Ucar, Mustafa H. B.; Albadwieh, Mohammad
• Is Part Of: Computer applications in engineering education. Volume 28:Issue 6(2020)
• Page Start: 1406
• Page End: 1420
• Publication Date: 2020-08-02
• Subjects: assessment -- computer‐based evaluation -- learning outcomes‐based analysis -- multiple‐choice questions -- question‐based analysis -- software tool; Engineering -- Study and teaching (Higher) -- Periodicals; Engineering -- Computer-assisted instruction -- Periodicals; 620
• DOI: 10.1002/cae.22309
• Languages: English
• ISSNs: 1061-3773

Abstract

Abstract: In this study, a computer‐based evaluation (CBE) software tool, namely CBE‐MCQs, is proposed to assist the instructor to evaluate and analyze traditional classroom‐based exams, which include multiple‐choice questions (MCQs) reliably, quickly, and automatically. Using the proposed CBE‐MCQs software tool while performing question analysis, the learning outcome‐based analyzes are performed by matching the learning outcomes of the courses with the exam questions. With question‐based evaluations, item analyzes such as item difficulty, item discrimination, and item distracters can be made easily. Also, with the question‐based evaluation, the instructor of the course can effectively measure the extent to which students achieve the learning outcomes of the related course. Thus, the CBE‐MCQs tool helps successful maintenance of both accreditation and training processes with the analysis it provides to the instructors. In addition, the final exam assessment of the course based on MCQs within the scope of the case study was carried out with the CBE‐MCQs software tool. Each MCQ was evaluated based on classical theory within the scope of difficulty index ( P ), discrimination index ( D ), and distracter efficiency ( DE ). It is observed that association between D and P was statistically significant according to Pearson correlation ( r = −0.600) and χ 2 ( χ 2 = 15.98, df = 8; α = 0.05). Also, a system usability survey of 10 statements was applied to the instructors using this software to evaluate the usability of the CBE‐MCQs software tool. As a result of a series of experimental studies and survey, the developed CBE‐MCQs software tool is evaluated as simple, effective, and satisfactory by 87.4% of the instructors.