TuneR: a framework for tuning software engineering tools with hands‐on instructions in R. Issue 6 (3rd May 2016)

• Record Type: Journal Article
• Authors: Borg, Markus
• Is Part Of: Journal of software. Volume 28:Issue 6(2016)
• Page Start: 427
• Page End: 459
• Publication Date: 2016-05-03
• Subjects: software engineering tools -- parameter tuning -- experiment framework -- empirical software engineering -- change impact analysis; Software engineering -- Periodicals; Computer software -- Development -- Periodicals; Software maintenance -- Periodicals; 005.1
• DOI: 10.1002/smr.1784
• Languages: English
• ISSNs: 2047-7473

Abstract

Abstract: Numerous tools automating various aspects of software engineering have been developed, and many of the tools are highly configurable through parameters. Understanding the parameters of advanced tools often requires deep understanding of complex algorithms. Unfortunately, suboptimal parameter settings limit the performance of tools and hinder industrial adaptation, but still few studies address the challenge of tuning software engineering tools. We present TuneR, an experiment framework that supports finding feasible parameter settings using empirical methods. The framework is accompanied by practical guidelines of how to use R to analyze the experimental outcome. As a proof‐of‐concept, we apply TuneR to tune ImpRec, a recommendation system for change impact analysis in a software system that has evolved for more than two decades. Compared with the output from the default setting, we report a 20.9% improvement in the response variable reflecting recommendation accuracy. Moreover, TuneR reveals insights into the interaction among parameters, as well as nonlinear effects. TuneR is easy to use, thus the framework has potential to support tuning of software engineering tools in both academia and industry. Copyright © 2016 John Wiley & Sons, Ltd. Abstract : Advanced software engineering tools often require complex tuning to perform well in a given context. Although suboptimal parameter settings hinder industrial adaptation, few studies provide guidelines. We present TuneR, an experiment framework that supports finding feasible parameter settings. The framework is accompanied by practical guidelines of how to use R to analyze the experimental outcome, revealing insights such as parameter interaction and nonlinear effects. As a proof‐of‐concept, we apply TuneR to tune ImpRec, a recommendation system for change impact analysis.