AAEE – Automated evaluation of students' essays in Arabic language. Issue 5 (September 2019)

• Record Type: Journal Article
• Authors: Azmi, Aqil M.; Al-Jouie, Maram F.; Hussain, Muhammad
• Is Part Of: Information processing & management. Volume 56:Issue 5(2019:Sep.)
• Page Start: 1736
• Page End: 1752
• Publication Date: 2019-09
• Subjects: Arabic -- Automatic essay scoring -- Latent semantic analysis -- Rhetorical structure theory -- Improving classroom teaching -- Interactive learning environments; Information storage and retrieval systems -- Periodicals; Information science -- Periodicals; Systèmes d'information -- Périodiques; Sciences de l'information -- Périodiques; Information science; Information storage and retrieval systems; Periodicals; 658.4038
• DOI: 10.1016/j.ipm.2019.05.008
• Languages: English
• ISSNs: 0306-4573

Abstract

Highlights: First system for automatically grading school children essays in Arabic. System modeled on grading scheme actually employed in Saudi Arabia, s schools. Relies on latent semantic analysis (LSA), and Rhetorical Structure Theory (RST). Our system achieved an accuracy of 90%. Correlation between system's auto-score and human judgement's scoring is 0.756. Abstract: Assessing student's essay writing and providing thoughtful feedback is a truly labor-intensive and time-consuming task. With human instructors already overwhelmed, the alternate is to consider a computer-based grading. Recent advances have generated renewed interest in automatic evaluation of essays (AEE). The AEEs instantaneous feedback and more consistent grading helps students draft better essays. This work presents a system to automatically grade the school children essays in Arabic, calling it AAEE for "automatic Arabic essays evaluator". The system is modeled upon the scoring scheme followed by the school instructors in Saudi Arabia. The instructors had specific criteria upon which an essay is assessed. Putting these criteria together we developed a system that relies on Latent Semantic Analysis, and Rhetorical Structure Theory. With this design we are able to assess individual components of the essay such as language proficiency, structure of the essay etc. To test the system, we collected essays by local school children covering grades 7–12. A total of 350 different handwritten essays—spanning eight different topics—each transcribed into computer readable format. The AAEE shows that 90% of the test essays were correctly scored, and a correlation of 0.756 between automatic and teachers' scoring. This exceeds the human-human correlation of 0.709 for the Arabic essays.