Automated black‐box testing of nominal and error scenarios in RESTful APIs. (23rd January 2022)

• Record Type: Journal Article
• Authors: Corradini, Davide; Zampieri, Amedeo; Pasqua, Michele; Viglianisi, Emanuele; Dallago, Michael; Ceccato, Mariano
• Other Names: Păsăreanu Corina guestEditor.; Zeller Andreas guestEditor.
• Is Part Of: Software testing, verification & reliability. Volume 32:Number 5(2022)
• Page Start: n/a
• Page End: n/a
• Publication Date: 2022-01-23
• Subjects: automatic test case generation -- black‐box testing -- REST APIs -- test oracle; Computer software -- Testing -- Periodicals; Computer software -- Verification -- Periodicals; Computer software -- Reliability -- Periodicals; 005.14
• DOI: 10.1002/stvr.1808
• Languages: English
• ISSNs: 0960-0833

Abstract

Abstract: RESTful APIs (or REST APIs for short) represent a mainstream approach to design and develop web APIs using the REpresentational State Transfer architectural style. Black‐box testing, which assumes only the access to the system under test with a specific interface, is the only viable option when white‐box testing is impracticable. This is the case for REST APIs: their source code is usually not (or just partially) available, or a white‐box analysis across many dynamically allocated distributed components (typical of a micro‐services architecture) is computationally challenging. This paper presents RestTestGen, a novel black‐box approach to automatically generate test cases for REST APIs, based on their interface definition (an OpenAPI specification). Input values and requests are generated for each operation of the API under test with the twofold objective of testing nominal execution scenarios and error scenarios. Two distinct oracles are deployed to detect when test cases reveal implementation defects. While this approach is mainly targeting the research community, it is also of interest to developers because, as a black‐box approach, it is universally applicable across different programming languages, or in the case external (compiled only) libraries are used in a REST API. The validation of our approach has been performed on more than 100 of real‐world REST APIs, highlighting the effectiveness of the approach in revealing actual faults in already deployed services. Abstract : The source code of REST APIs is usually not (or just partially) available, or difficult to analyse because spread across many dynamically allocated distributed components (typical in micro‐services architectures), or due to external (compiled only) libraries or frameworks. RestTestGen is a novel black‐box approach to automatically generate test cases for REST APIs, based on their interface definition (an OpenAPI specification). Input values and requests are generated for each operation of the API under test with the twofold objective of testing nominal execution scenarios and error scenarios. RestTestGen has been successfully applied to test more than 100 of real‐world REST APIs, highlighting the effectiveness of the approach in revealing actual faults in already deployed services.