Using Technical Cybersecurity Exercises in University Admissions and Skill Evaluation. Issue 19 (2019)

• Record Type: Journal Article
• Authors: Maennel, Kaie; Mäses, Sten; Sütterlin, Stefan; Ernits, Margus; Maennel, Olaf
• Is Part Of: IFAC-PapersOnLine. Volume 52:Issue 19(2019)
• Page Start: 169
• Page End: 174
• Publication Date: 2019
• Subjects: Cybersecurity -- university selection process -- skills assessment -- virtual labs; Automatic control -- Periodicals; 629.805
• DOI: 10.1016/j.ifacol.2019.12.169
• Languages: English
• ISSNs: 2405-8963

Abstract

Abstract: Cybersecurity is a fast growing domain. The supply of workforce entering the labour market can not match the current demands. Due to this currently existing and predicted future skills gap in the labour market, educational institutions attempt to minimize dropouts and study times. As a direct consequence, the relevance of valid admission and selection procedures has grown in recent years. However, there is a mismatch between the increased demand for high-quality admission procedures and the still existing lack of established methods and routines to conduct these. In this paper we discuss our experience from running admissions in one of the oldest European master level cybersecurity curricula in Europe. We argue that cybersecurity skills assessment cannot simply be traditional knowledge-based assessments as this may exclude suitable candidates, who have not had the opportunity to learn the subject matter or are joining from different fields. Also selection decision cannot be done purely based on previous grades, because decomposing school subjects into cybersecurity skills is challenging due to the domain's interdisciplinary nature. We present a technical skills assessment method using cloud-based virtual labs that can be done by the candidates remotely. Those labs focus on assessing the technical competencies of a candidate and leave the assessment of non-technical skills (which are at least equally important) to a human interviewer. Also identifying cheaters, who do not prepare their labs themselves, will be left for the human interviewer. Such on-line exercises show potential as scalable option to evaluate the cybersecurity technical skills, motivational levels and cognitive strategies applied for problem-solving in a complex, novel task when being under performance pressure. The lessons learned are shared; feedback obtained from the applicants and possible technical metrics for predicting their success in a cybersecurity program are explored. As further work, we plan to conduct full data analysis and time-delayed interviews to generate hypothesis that can be further empirically tested with appropriate designs to detect causal relationships.