Performance, Usability, and User Experience of Rayyan for Systematic Reviews. Issue 1 (14th October 2022)

• Record Type: Journal Article
• Authors: Yu, Fei; Liu, Chuanrui; Sharmin, Selina
• Is Part Of: Proceedings of the Association for Information Science and Technology. Volume 59:Issue 1(2022)
• Page Start: 843
• Page End: 844
• Publication Date: 2022-10-14
• Subjects: Performance -- Rayyan -- Systematic Reviews -- Usability -- User experience; Information science -- Congresses; Information technology -- Congresses; Information science; Information technology; Conference papers and proceedings; 020
• DOI: 10.1002/pra2.745
• Languages: English
• ISSNs: 2373-9231

Abstract

ABSTRACT: This study investigated performance, usability, and user experience with Rayyan, a systematic review (SR) tool. A remote usability inspection was conducted via the institutional Zoom platform. Each participant was randomly assigned to two groups and completed two citation screening sessions independently. Data were collected through pre‐ and post‐test questionnaires, screening sessions, and Systems Usability Scale (SUS) surveys. Thirteen participants completed the study. We found significant accuracy difference between Web Rayyan and its mobile app (t[12] = 1.877 p = .042), but no statistically sequential effect on screening performance. The average SUS scores of both Rayyan versions were below 68. More participants (77%) preferred Web Rayyan to the mobile app. The mobile Rayyan has potential if its user interface design can prevent errors or allow easy action reversal. As the first controlled experiment investigating both Web and mobile Rayyan, this study provides empirical evidence to guide the design, development, and selection of SR tools.