Evaluating and comparing language workbenches: Existing results and benchmarks for the future. (December 2015)

• Record Type: Journal Article
• Authors: Erdweg, Sebastian; van der Storm, Tijs; Völter, Markus; Tratt, Laurence; Bosman, Remi; Cook, William R.; Gerritsen, Albert; Hulshout, Angelo; Kelly, Steven; Loh, Alex; Konat, Gabriël; Molina, Pedro J.; Palatnik, Martin; Pohjonen, Risto; Schindler, Eugen; Schindler, Klemens; Solmi, Riccardo; Vergu, Vlad; Visser, Eelco; van der Vlist, Kevin; Wachsmuth, Guido; van der Woning, Jimi
• Is Part Of: Computer languages, systems & structures. Volume 44:Part A(2015)
• Page Start: 24
• Page End: 47
• Publication Date: 2015-12
• Subjects: Language workbenches -- Domain-specific languages -- Questionnaire language -- Survey -- Benchmarks; Programming languages (Electronic computers) -- Periodicals; Computer networks -- Periodicals; Computer architecture -- Periodicals; Computer systems -- Periodicals; Langage de programmation; Réseau d'ordinateurs; Architecture d'ordinateur; Périodique électronique (Descripteur de forme); Ressource Internet (Descripteur de forme); 005.13
• DOI: 10.1016/j.cl.2015.08.007
• Languages: English
• ISSNs: 1477-8424

Abstract

Abstract: Language workbenches are environments for simplifying the creation and use of computer languages. The annual Language Workbench Challenge (LWC) was launched in 2011 to allow the many academic and industrial researchers in this area an opportunity to quantitatively and qualitatively compare their approaches. We first describe all four LWCs to date, before focussing on the approaches used, and results generated, during the third LWC. We give various empirical data for ten approaches from the third LWC. We present a generic feature model within which the approaches can be understood and contrasted. Finally, based on our experiences of the existing LWCs, we propose a number of benchmark problems for future LWCs. Abstract : Highlights: We describe the challenges posed by the 4 Language Workbench Challenges run so far. We establish a feature model that captures the design space of language workbenches. We classify 10 language workbenches according to our feature model. We present empirical data on 10 implementations of a questionnaire DSL. We propose benchmark problems to be used in future Language Workbench Challenges.