Insights about large-scale online peer assessment from an analysis of an astronomy MOOC. (October 2017)

• Record Type: Journal Article
• Authors: Formanek, Martin; Wenger, Matthew C.; Buxner, Sanlyn R.; Impey, Chris D.; Sonam, Tenzin
• Is Part Of: Computers & education. Volume 113(2017)
• Page Start: 243
• Page End: 262
• Publication Date: 2017-10
• Subjects: Peer assessment -- MOOCs -- Online learning -- Informal learning; Education -- Data processing -- Periodicals; Education -- Periodicals; Computers -- Periodicals; Computer-Assisted Instruction -- Periodicals; Éducation -- Informatique -- Périodiques; Electronic journals; 370.285
• DOI: 10.1016/j.compedu.2017.05.019
• Languages: English
• ISSNs: 0360-1315

Abstract

Abstract: In this work we investigate the peer grading assignments which were an integral part of the astronomy Massive Open Online Course (MOOC) (Astronomy: Exploring Time and Space) provided through Coursera from March to May 2015. Our general goal is to assess the role of peer graded assignments in such courses and how they contribute to students' learning and motivation. In order to achieve this broad goal we look at the peer grading process from multiple perspectives. We present an analysis of demographics for peer grading participants and show how they are different from the general course population. We also look at different aspects of peer grading assignments such as lengths of essays, time spent grading, number of gradings performed, final grades and percentage of relevant videos watched. We compare these distributions for different assignments and also their correlations on a level of individual learners. We show that participation in the first peer graded assignment is the best predictor of completion for the course as a whole. Moreover, learners who did well on the first peer graded assignment show better engagement and do better in the course overall. Finally, we report on validity and reliability of peer graders as compared to instructor graders and trained undergraduate graders. Highlights: Peer grading established as a main predictor of MOOC course completion. Peer grading participants are different from the general course population. Patterns of the first peer grading assignment were different. Peer grading results strongly correlate with other effort invested in the course.