ProvBench: A performance provenance capturing framework for heterogeneous research computing environments. (10th January 2022)

• Record Type: Journal Article
• Authors: Liu, Fang (Cherry); Belgin, Mehmet; Zhang, Nuyun; Manalo, Kevin; Lara, Ruben; Stone, Christopher P.; Manno, Paul
• Is Part Of: Concurrency and computation. Volume 34:Number 10(2022)
• Page Start: n/a
• Page End: n/a
• Publication Date: 2022-01-10
• Subjects: computing system -- data analysis -- performance benchmark -- software tool; Parallel processing (Electronic computers) -- Periodicals; Parallel computers -- Periodicals; 004.35
• DOI: 10.1002/cpe.6820
• Languages: English
• ISSNs: 1532-0626

Abstract

Abstract: This article presents a benchmarking framework, namely "ProvBench, " with a specific focus on provenance of collected data, capable of identifying and measuring the impact of changes to hardware, operating system, software, middleware, and services that constitute a highly complex and heterogeneous research computing environment. The provenance is retained via detailed and automated recording of hardware details, runtime environment, software and libraries used, input data and results, as well as execution logs of the computation. This capability is particularly essential for constant monitoring and fast identification of abnormalities. The framework is compatible across different operating systems and varied software environments that support software modules. Its modular object‐oriented design allows for easy expansions, that is, adding new software and tests is straightforward. ProvBench is being actively used in our center for testing acquired equipment, evaluation of preproduction systems, assessing the impact of system and software changes, finding bad nodes, and other useful purposes with successful results.