Remote proctoring: Lessons learned from the COVID‐19 pandemic effect on the large scale on‐line assessment at Tel Aviv University. (30th September 2022)

• Record Type: Journal Article
• Authors: Patael, Smadar; Shamir, Julia; Soffer, Tal; Livne, Eynat; Fogel‐Grinvald, Haya; Kishon‐Rabin, Liat
• Is Part Of: Journal of computer assisted learning. Volume 38:Number 6(2022)
• Page Start: 1554
• Page End: 1573
• Publication Date: 2022-09-30
• Subjects: COVID‐19 -- Higher education -- On‐line proctored assessment -- Remote proctoring; Computer-assisted instruction -- Periodicals; 371.334
• DOI: 10.1111/jcal.12746
• Languages: English
• ISSNs: 0266-4909

Abstract

Abstract: Background: The global COVID‐19 pandemic turned the adoption of on‐line assessment in the institutions for higher education from possibility to necessity. Thus, in the end of Fall 20/21 semester Tel Aviv University (TAU)—the largest university in Israel—designed and implemented a scalable procedure for administering proctored remote examinations. This procedure is applicable to different kinds of examinations in diverse disciplines and to disparate degree levels. Objectives: This study aims to deepen the knowledge with respect to the design and adoption of remote proctoring at an institutional level Methods: First, based on lessons learned from the first semester of COVID‐19, we describe the development and implementation of an institution‐wide protocol for conducting on‐line proctored assessment. We show the large‐scale applicability of the protocol for administering examinations via Moodle with remote proctoring via Zoom. The accurately designed procedures that included careful consideration of all parties involved—academic staff members, students, administrative staff and proctors—enabled TAU to successfully assess the learning outcomes of its 25, 000 students, while maintaining validity, compatibility and reliability of the assessment. Second, we report combined data from surveys of 4380 students and 188 faculty‐members (18% and 12% response rate respectively), conducted at TAU following these examinations. Results and Conclusions: With over 90% of the students experiencing at least one, and 80% of faculty‐members administering at least one proctored remote examination, these heterogeneous sources of data allow us to investigate a unique and complementary perspective of the process. Some of the significant findings consist of the over‐all perception of the students' integrity by both groups surveyed; the discrepancy in the groups' perspectives of the adequate form of learning‐assessment—with the instructors preferring proctored examinations and the students—formative‐assessment; and the influence of the pedagogical challenges on the perception of validity that outweighed other factors, for example, the pandemic‐related stress, the on‐line assessment challenges and other. Lay Description: What is already known about this topic: Proctoring exams remotely is challenging, due to technical, ethical and other concerns. Proctoring can be conducted remotely by a human, record and review later technology, automated algorithms to detect atypical behaviour. Different methods for remote proctoring in higher education have been implemented on small scale. What this paper adds: Review of studies examining various facets of remote proctoring before and during COVID‐19. A protocol for remote proctoring in a large‐scale university, based on the first post COVID‐19 semester. The perspectives of students and faculty members regarding testing with remote proctoring. Implications for future practice and/or policy: The protocol can assist higher education institutions in developing their own guidelines. The implementation should take into consideration ethical norms and constitutional limitations. Unlike automated process using algorithms, remote human proctoring is not widely opposed by students. Further research comparing remotely proctored exam performance with the f2f proctoring is needed.