Towards repeatability & verifiability in networking experiments: A stochastic framework. (1st March 2017)

• Record Type: Journal Article
• Authors: Sharma, Swati; Hussain, Alefiya; Saran, Huzur
• Is Part Of: Journal of network and computer applications. Volume 81(2017)
• Page Start: 12
• Page End: 23
• Publication Date: 2017-03-01
• Subjects: Experiment characterization -- Verifiability -- Repeatability -- Event-based Markov chain characterization -- Time-series characterization; Microcomputers -- Periodicals; Computer networks -- Periodicals; Application software -- Periodicals; Micro-ordinateurs -- Périodiques; Réseaux d'ordinateurs -- Périodiques; Logiciels d'application -- Périodiques; Application software; Computer networks; Microcomputers; Periodicals; 004.05; 004
• DOI: 10.1016/j.jnca.2016.07.001
• Languages: English
• ISSNs: 1084-8045

Abstract

Abstract: Growing complexity of networking experiments has rendered the accuracy, repetition and reproduction of experimental results indispensable. This research work aims at developing a tool to facilitate repeatability and verifiability of experiments on diverse experimentation platforms. The process involves studying an experiment with respect to its constituent events and creating time-series based and event based characterizations from logs and measurement data. The framework thus developed was applied to 6000 experiment trials comprising of 1500 DETER testbed DNS-cache poisoning attack, 3200 ns2 web-traffic generation, 1000 real-time web-traffic generation and 300 ns3 wireless mobility experiment trials. We observed that these characterizations were sensitive to experiment configuration. We also investigated the impact of variations in topology, traffic, hardware and mobility on them (characterizations). Differences in characterizations were quantified by standard distance measures: KL Divergence, Total Variation Distance and Euclidean Distance. Our results were statistically verified by hypothesis testing and analysis through ANOVA, chi-square and correlation tests. Thus, our framework provides a direct and precise method to compare two executions of a stochastic networking experiment for simulations, emulation-based testbeds and real-time network experiments. These results form the groundwork for generating a validity management framework and will help to achieve verifiability in experiment executions.