Performance regression testing and run-time verification of components in robotics systems. (17th November 2017)

• Record Type: Journal Article
• Authors: Wienke, J.; Wrede, S.
• Is Part Of: Advanced robotics. Volume 31:Number 22(2017)
• Page Start: 1177
• Page End: 1192
• Publication Date: 2017-11-17
• Subjects: Resource utilization -- testing -- performance bugs -- continuous integration -- resource prediction -- performance regressions; Robotics -- Periodicals; Robotics -- Japan -- Periodicals; Robotics; Japan; Periodicals; 629.89205
• DOI: 10.1080/01691864.2017.1395360
• Languages: English
• ISSNs: 0169-1864

Abstract

Abstract: Unintended changes in the utilization of resources like CPU and memory can lead to severe problems for the operation of robotics and intelligent systems. Nevertheless, systematic testing for such performance regressions has largely been ignored in this domain. We present a method to specify and execute performance tests for individual components of component-based robotics systems based on their middleware interfaces. The method includes an automatic analysis of each component revision against previous ones that reports potential changes to the resource utilization characteristics. This informs developers about the impact of their changes. We describe the design of the framework and present evaluation results for the automatic detection of performance changes based on tests for a variety of robotics components. Additionally, we demonstrate how performance tests can be used as a basis for learning resource utilization models of components. These models can be used to detect faults at system run-time, which provides an additional level of safety for the systems besides the offline testing. Abstract : Presents a testing framework for automatic performance regression detection. Individual components are tested based on their middleware interfaces. Test recordings can be used to train resource prediction models.