Independent markers of master's theses show low levels of agreement. Issue 5 (4th July 2019)

• Record Type: Journal Article
• Authors: Williams, Logan; Kemp, Simon
• Is Part Of: Assessment & evaluation in higher education. Volume 44:Issue 5(2019)
• Page Start: 764
• Page End: 771
• Publication Date: 2019-07-04
• Subjects: Academic peer review -- reliability -- Master's thesis -- grades; Education, Higher -- Evaluation -- Periodicals; 378
• DOI: 10.1080/02602938.2018.1535052
• Languages: English
• ISSNs: 0260-2938

Abstract

Abstract: We examined the reliability of grading master's theses at a New Zealand university, where a variant of the academic journal review system is employed. The overall correlation between the grades recommended by internal and external markers of master's theses in psychology and applied psychology at this university was 0.39, which is similar to that produced between reviewers of submissions to academic journals. We conclude that thesis grading is about as reliable as the determination of the merit of journal submissions, which is usually regarded as rather poor. Applying a more objective rubric to thesis marking might raise the reliability, but such a change might also lower the creative and unique nature of theses, and hence reduce the contribution that an individual thesis can make to its particular discipline.