Improving fuzzing assessment methods through the analysis of metrics and experimental conditions. Issue 124 (January 2023)

• Record Type: Journal Article
• Authors: Eceiza, Maialen; Flores, Jose Luis; Iturbe, Mikel
• Is Part Of: Computers & security. Issue 124(2023)
• Publication Date: 2023-01
• Subjects: Fuzzing -- Evaluation methodology -- Security -- Software testing -- Metrics; Computer security -- Periodicals; Electronic data processing departments -- Security measures -- Periodicals; 005.805
• DOI: 10.1016/j.cose.2022.102946
• Languages: English
• ISSNs: 0167-4048

Abstract

Abstract: Fuzzing is nowadays one of the most widely used bug hunting techniques. By automatically generating malformed inputs, fuzzing aims to trigger unwanted behavior on its target. While fuzzing research has matured considerably in the last years, the evaluation and comparison of different fuzzing proposals remain challenging, as no standard set of metrics, data, or experimental conditions exist to allow such observation. This paper aims to fill that gap by proposing a standard set of features to allow such comparison. For that end, it first reviews the existing evaluation methods in the literature and discusses all existing metrics by evaluating seven fuzzers under identical experimental conditions. After examining the obtained results, it recommends a set of practices –particularly on the metrics to be used–, to allow proper comparison between different fuzzing proposals.