Towards reducing the time needed for load testing. Issue 3 (16th July 2020)

• Record Type: Journal Article
• Authors: AlGhamdi, Hammam M.; Bezemer, Cor‐Paul; Shang, Weiyi; Hassan, Ahmed E.; Flora, Parminder
• Is Part Of: Journal of software. Volume 35:Issue 3(2023)
• Page Start: n/a
• Page End: n/a
• Publication Date: 2020-07-16
• Subjects: load testing -- performance analysis -- performance testing; Software engineering -- Periodicals; Computer software -- Development -- Periodicals; Software maintenance -- Periodicals; 005.1
• DOI: 10.1002/smr.2276
• Languages: English
• ISSNs: 2047-7473

Abstract

Abstract: The performance of large‐scale systems must be thoroughly tested under various levels of workload, as load‐related issues can have a disastrous impact on the system. However, load testing often requires a large amount of time, running from hours to even days. In our prior work, we reduced the execution time of a load test by detecting repetitiveness in individual performance metric values, such as CPU utilization, that are observed during the test. However, as we explain in this paper, disregarding combinations of performance metrics may miss important information about the load‐related behavior of a system. In this paper we revisit our prior approach, by proposing an approach that reduces the execution time of a load test by detecting whether a test no longer exercises new combinations of the observed performance metrics. We study three open source systems, in which we use our new and prior approaches to reduce the execution time of a 24‐hour load test. We show that our new approach is capable of reducing the execution time of the test to less than 8.5 hours, while preserving a coverage of at least 95% of the combinations that are observed between the performance metrics during the 24‐hour tests. Abstract : The performance of large‐scale systems must be thoroughly tested under various levels of workload, as load‐related issues can have a disastrous impact on the system. However, load tests often require a large amount of time, running from hours to even days, to execute. In our prior work, we proposed to reduce the execution time of a load test by detecting repetitiveness in combinations of performance metric values, such as CPU utilization or memory usage, that are observed during the test..