Reducing the maintenance effort for parameterization of representative load tests using annotations. (16th September 2019)

• Record Type: Journal Article
• Authors: Schulz, Henning; van Hoorn, André; Wert, Alexander
• Other Names: Saadatmand Mehrdad guestEditor.; Lindström Birgitta guestEditor.; Aichernig Bernhard K. guestEditor.
• Is Part Of: Software testing, verification & reliability. Volume 30:Number 1(2020)
• Page Start: n/a
• Page End: n/a
• Publication Date: 2019-09-16
• Subjects: load test parameterization -- load testing -- workload evolution -- workload model extraction; Computer software -- Testing -- Periodicals; Computer software -- Verification -- Periodicals; Computer software -- Reliability -- Periodicals; 005.14
• DOI: 10.1002/stvr.1712
• Languages: English
• ISSNs: 0960-0833

Abstract

Summary: Directly affecting the user experience, performance is a crucial aspect of today's software applications. Representative load testing allows to effectively test and preserve the performance before delivery by mimicking the actually expected workload. In the literature, various approaches have been proposed for extracting representative load tests from recorded user sessions. However, these approaches require manual parameterization for specifying input data and adjusting static properties such as a request's domain name. This manual effort accumulates when load tests need to be updated due to changing production workloads and APIs. In this paper, we address the reduction of the maintenance effort for representative load testing. We introduce input data and properties annotations (IDPAs) that store manual parameterizations and can be evolved automatically. Experts only have to parameterize extracted load tests initially. For dealing with API changes, we develop approaches to evolve IDPAs for the types of changes described in the literature. We evaluated our approach in two experimental studies, by deriving effort estimation models, and in an industrial case study including four different software projects. Our evaluation shows that IDPAs can parameterize generated load tests for restoring the representativeness, especially for applications with workloads dominated by request orders and rates. The maintenance effort can be reduced from a quadratic cumulative effort over time to a linear cumulative effort for a typical mix of API changes. Furthermore, we were able to express all parameterizations required by the industrial projects using the IDPA but also had to integrate extensions using the provided extension mechanisms. Abstract : Existing approaches allow to extract representative load tests from recorded user sessions automatically, but require manual parameterization of the tests. We propose an approach for automating the parameterization as well by storing input data specifications and property adjustments in a separate annotation model. By evolving the model over workload and API changes, we can reduce the maintenance effort from a quadratic cumulative effort to a linear one as a function of time.