Best practices of testing database manipulation code. Issue 111 (January 2023)

• Record Type: Journal Article
• Authors: Gobert, Maxime; Nagy, Csaba; Rocha, Henrique; Demeyer, Serge; Cleve, Anthony
• Is Part Of: Information systems. Issue 111(2023)
• Publication Date: 2023-01
• Subjects: Testing -- Database manipulation code -- Empirical study; Database management -- Periodicals; Electronic data processing -- Periodicals; Bases de données -- Gestion -- Périodiques; Informatique -- Périodiques; Database management; Electronic data processing; Periodicals; 005.7
• DOI: 10.1016/j.is.2022.102105
• Languages: English
• ISSNs: 0306-4379

Abstract

Abstract: Software testing enables development teams to maintain the quality of a software system while it evolves. The database manipulation code requires special attention in this context. However, it is often neglected and suffers from software maintenance problems. In this paper, we study the current state-of-the-practice in testing database manipulation code. We first analysed the tests of 72 open-source projects to gain insight into the coverage of database access code. The database was poorly tested: 46% of the projects did not cover with tests half of their database access methods, and 33% did not cover the database code at all. This poor coverage motivated us to study developers' challenges and best practices. (i) First, we analysed 532 questions on Stack Exchange sites and deduced a taxonomy of issues . Developers mostly looked for general best practices to test database access code. Their technical questions were related to database management, mocking, parallelisation, or framework/tool usage. (ii) Next, we examined the answers to these questions. We manually labelled 598 answers to 255 questions. We distinguished 363 solutions and organised them in a taxonomy of best practices . Most of the suggestions considered the testing environment and recommended various tools or configurations. The second largest category was database management, where many addressed database initialisation and clean-up between tests. Other categories pertained to code structure or design, concepts, performance, processes, test characteristics, test code, and mocking. We illustrate the two taxonomies through intriguing examples. Highlights: A study on the coverage of database access code in the tests of open-source projects. A qualitative assessment of developers' issues when testing database access code. An investigation of developers' best practices when testing database access code. Taxonomies of issues and best practices when testing database access code. The full dataset of the studies is publicly available as an online appendix.