Improving test automation maturity: A multivocal literature review. (15th February 2022)
- Record Type:
- Journal Article
- Title:
- Improving test automation maturity: A multivocal literature review. (15th February 2022)
- Main Title:
- Improving test automation maturity: A multivocal literature review
- Authors:
- Wang, Yuqing
Mäntylä, Mika V.
Liu, Zihao
Markkula, Jouni
Raulamo‐jurvanen, Päivi - Abstract:
- Abstract: Mature test automation is key for achieving software quality at speed. In this paper, we present a multivocal literature review with the objective to survey and synthesize the guidelines given in the literature for improving test automation maturity. We selected and reviewed 81 primary studies, consisting of 26 academic literature and 55 grey literature sources. From primary studies, we extracted 26 test automation best practices (e.g., Define an effective test automation strategy, Set up good test environments, and Develop high‐quality test scripts) and collected many pieces of advice (e.g., in forms of implementation/improvement approaches, technical techniques, concepts, and experience‐based heuristics) on how to conduct these best practices. We made main observations: (1) There are only six best practices whose positive effect on maturity improvement have been evaluated by academic studies using formal empirical methods; (2) several technical related best practices in this MLR were not presented in test maturity models; (3) some best practices can be linked to success factors and maturity impediments proposed by other scholars; (4) most pieces of advice on how to conduct proposed best practices were identified from experience studies and their effectiveness need to be further evaluated with cross‐site empirical evidence using formal empirical methods; (5) in the literature, some advice on how to conduct certain best practices are conflicting, and some advice onAbstract: Mature test automation is key for achieving software quality at speed. In this paper, we present a multivocal literature review with the objective to survey and synthesize the guidelines given in the literature for improving test automation maturity. We selected and reviewed 81 primary studies, consisting of 26 academic literature and 55 grey literature sources. From primary studies, we extracted 26 test automation best practices (e.g., Define an effective test automation strategy, Set up good test environments, and Develop high‐quality test scripts) and collected many pieces of advice (e.g., in forms of implementation/improvement approaches, technical techniques, concepts, and experience‐based heuristics) on how to conduct these best practices. We made main observations: (1) There are only six best practices whose positive effect on maturity improvement have been evaluated by academic studies using formal empirical methods; (2) several technical related best practices in this MLR were not presented in test maturity models; (3) some best practices can be linked to success factors and maturity impediments proposed by other scholars; (4) most pieces of advice on how to conduct proposed best practices were identified from experience studies and their effectiveness need to be further evaluated with cross‐site empirical evidence using formal empirical methods; (5) in the literature, some advice on how to conduct certain best practices are conflicting, and some advice on how to conduct certain best practices still need further qualitative analysis. Abstract : This paper reviewed 81 primary studies (26 academic literature and 55 grey literature sources). It extracted 26 test automation best practices (e.g., define an effective test automation strategy, set up good test environments, and develop high‐quality test scripts) and collected many pieces of advice (e.g., in forms of implementation/improvement approaches, technical techniques, concepts, experience‐based heuristics) on how to conduct these best practices. … (more)
- Is Part Of:
- Software testing, verification & reliability. Volume 32:Number 3(2022)
- Journal:
- Software testing, verification & reliability
- Issue:
- Volume 32:Number 3(2022)
- Issue Display:
- Volume 32, Issue 3 (2022)
- Year:
- 2022
- Volume:
- 32
- Issue:
- 3
- Issue Sort Value:
- 2022-0032-0003-0000
- Page Start:
- n/a
- Page End:
- n/a
- Publication Date:
- 2022-02-15
- Subjects:
- improvement -- maturity -- practice -- software -- systematic literature review -- test automation
Computer software -- Testing -- Periodicals
Computer software -- Verification -- Periodicals
Computer software -- Reliability -- Periodicals
005.14 - Journal URLs:
- http://onlinelibrary.wiley.com/ ↗
- DOI:
- 10.1002/stvr.1804 ↗
- Languages:
- English
- ISSNs:
- 0960-0833
- Deposit Type:
- Legaldeposit
- View Content:
- Available online (eLD content is only available in our Reading Rooms) ↗
- Physical Locations:
- British Library DSC - 8321.457500
British Library DSC - BLDSS-3PM
British Library STI - ELD Digital store - Ingest File:
- 21262.xml