A verified and optimized Stream X‐Machine testing method, with application to cloud service certification. (18th February 2020)

• Record Type: Journal Article
• Authors: Simons, Anthony J. H.; Lefticaru, Raluca
• Is Part Of: Software testing, verification & reliability. Volume 30:Number 3(2020)
• Page Start: n/a
• Page End: n/a
• Publication Date: 2020-02-18
• Subjects: cloud computing -- cloud service broker -- functional testing -- service certification -- specification -- state‐based testing -- test grounding -- verification -- X‐machines; Computer software -- Testing -- Periodicals; Computer software -- Verification -- Periodicals; Computer software -- Reliability -- Periodicals; 005.14
• DOI: 10.1002/stvr.1729
• Languages: English
• ISSNs: 0960-0833

Abstract

Summary: The Stream X‐Machine (SXM) testing method provides strong and repeatable guarantees of functional correctness, up to a specification. These qualities make the method attractive for software certification, especially in the domain of brokered cloud services, where arbitrage seeks to substitute functionally equivalent services from alternative providers. However, practical obstacles include the difficulty in providing a correct specification, the translation of abstract paths into feasible concrete tests and the large size of generated test suites. We describe a novel SXM verification and testing method, which automatically checks specifications for completeness and determinism, prior to generating complete test suites with full grounding information. Three optimization steps achieve up to a 10‐fold reduction in the size of the test suite, removing infeasible and redundant tests. The method is backed by a set of tools to validate and verify the SXM specification, generate technology‐agnostic test suites and ground these in SOAP, REST or rich‐client service implementations. The method was initially validated using seven specifications, three cloud platforms and five grounding strategies. Abstract : This article describes a novel Stream X‐Machine method for verifying specifications and then generating complete functional test suites, targeting software certification in the cloud. The accompanying toolset supports validation, verification, and test generation from a technology‐agnostic specification, grounding tests for SOAP, REST, or rich‐client service implementations. The verification and testing method supports three optimizations to remove redundant and infeasible test paths and compress generated test suites, achieving a tenfold reduction in test suite size without loss of conformance testing power.