ExVivoMicroTest: ExVivo Testing of Microservices. Issue 4 (12th April 2022)

• Record Type: Journal Article
• Authors: Gazzola, Luca; Goldstein, Maayan; Mariani, Leonardo; Mobilio, Marco; Segall, Itai; Tundo, Alessandro; Ussi, Luca
• Other Names: Bertolino Antonia guestEditor.; Hong Shin guestEditor.; Mathur Aditya P. guestEditor.
• Is Part Of: Journal of software. Volume 35:Issue 4(2023)
• Page Start: n/a
• Page End: n/a
• Publication Date: 2022-04-12
• Subjects: ex vivo testing -- microservices -- regression testing -- software testing; Software engineering -- Periodicals; Computer software -- Development -- Periodicals; Software maintenance -- Periodicals; 005.1
• DOI: 10.1002/smr.2452
• Languages: English
• ISSNs: 2047-7473

Abstract

Abstract: Microservice‐based applications consist of multiple services that can evolve independently. When a service must be updated, it is first tested with in‐house regression test suites. However, the test suites that are executed are usually designed without the exact knowledge about how the services will be accessed and used in the field; therefore, they may easily miss relevant test scenarios, failing to prevent the deployment of faulty services. To address this problem, we introduce ExVivoMicroTest, an approach that analyzes the execution of deployed services at run‐time in the field, in order to generate test cases for future versions of the same services. ExVivoMicroTest implements lightweight monitoring and tracing capabilities, to inexpensively record executions that can be later turned into regression test cases that capture how services are used in the field. To prevent accumulating an excessive number of test cases, ExVivoMicroTest uses a test coverage model that can discriminate the recorded executions between the ones that are worth to be turned into test cases and the ones that should be discarded. The resulting test cases use a mocked environment that fully isolates the service under test from the rest of the system to faithfully reply interactions. We assessed ExVivoMicroTest with the PiggyMetrics and Train Ticket open source microservice applications and studied how different configurations of the monitoring and tracing logic impact on the capability to generate test cases. Abstract : This paper proposes ExVivoMicroTest, a technique that can trace and turn interactions between microservices recorded in the field into in‐house regression test cases. ExVivoMicroTest includes policies to dynamically adjust tracing probabilities on the collected data, to collect interactions only when they are likely to represent new untested behaviors. Empirical results show that ExVivoMicroTest can generate regression test suites that cover a relevant portion of the behavioral space of the microservices under tests.