An Empirical Study on How Sapienz Achieves Coverage and Crash Detection. Issue 4 (5th December 2021)

• Record Type: Journal Article
• Authors: Arcuschin, Iván; Galeotti, Juan Pablo; Garbervetsky, Diego
• Other Names: Bertolino Antonia guestEditor.; Hong Shin guestEditor.; Mathur Aditya P. guestEditor.
• Is Part Of: Journal of software. Volume 35:Issue 4(2023)
• Page Start: n/a
• Page End: n/a
• Publication Date: 2021-12-05
• Subjects: Android -- empirical study -- Evolutionary Algorithms -- Random Search -- Sapienz -- test generation; Software engineering -- Periodicals; Computer software -- Development -- Periodicals; Software maintenance -- Periodicals; 005.1
• DOI: 10.1002/smr.2411
• Languages: English
• ISSNs: 2047-7473

Abstract

Abstract: Several tools for automatically testing Android applications have been proposed. In particular, Sapienz is a search‐based tool that has been recently deployed in an industrial setting. Although it has been shown that Sapienz outperforms several state‐of‐the‐art tools, it is still to be seen what features of SAPIENZ impact the most on its effectiveness. We conducted an extensive empirical study where we compare the impact of the search algorithm and the usage of motif genes, a more compact representation of individuals. Our empirical study shows that the usage of motif genes improves coverage both for Evolutionary Algorithms and random approaches. In particular, it also shows that NSGA‐II, the multi‐objective evolutionary algorithm used by Sapienz, does not have a clear improvement over other algorithms. In terms of number of crashes detected, our study shows that both NSGA‐II and Random Search perform similarly. While the usage of motif genes improves the crash detection of algorithms, it is not enough to make it statistically significant. These facts cast doubts about the use of Evolutionary Algorithms in the context of Android test generation and suggest that motif genes can have a great impact on the overall effectiveness. Abstract : We present an empirical study on the tool Sapienz for Android test generation, where we compare the impact of the search algorithm and the usage of motif genes, a more compact representation of individuals. Our study shows that usage of motif genes improves statement coverage both for evolutionary algorithms and random approaches. In particular, it also shows that NSGA‐II, the multi‐objective evolutionary algorithm used by Sapienz, does not have a clear improvement over other algorithms.