How to write a plugin to export job, power, energy, and system environmental data from your Cray® XC™ system. (25th September 2017)

• Record Type: Journal Article
• Authors: Martin, Steven; Whitney, Cary; Rush, David; Kappel, Matthew
• Other Names: Xhafa Fatos guestEditor.; Michael Scott guestEditor.; He Yun guestEditor.
• Is Part Of: Concurrency and computation. Volume 30:Number 1(2018)
• Page Start: n/a
• Page End: n/a
• Publication Date: 2017-09-25
• Subjects: Cray XC40 -- energy efficiency -- power measurement -- power monitoring; Parallel processing (Electronic computers) -- Periodicals; Parallel computers -- Periodicals; 004.35
• DOI: 10.1002/cpe.4299
• Languages: English
• ISSNs: 1532-0626

Abstract

Summary: In this paper, we take a deep dive into writing a plugin to export power, energy, and other system environmental data from a Cray ® XC ™ system. With the release of the System Management Workstation 8.0 software, Cray has enabled customers to create site‐specific plugins to export all of the data that can flow into the Cray Power Management Database into site‐specific infrastructure. In this paper, we give practical information on what data are available using the plugin, and how to write, test, and deploy a plugin. We also share and explain example plugin code, detail design considerations when architecting a plugin, and look at some practical use cases supported by exporting telemetry data off a Cray ® XC ™ system. This paper is targeted at plugin developers, system administrators, data scientists, and site planners. The plugin feature was developed by Cray in response to discussions with and requirements from select members of the CUG XTreme SIG. This paper demonstrates lessons learned developing prototype plugins that export data off Cray ® XC ™ systems using Kafka, Redis Pub/Sub, and RabbitMQ. This plugin capability is in‐use internally at Cray, used in production at NERSC, and is under consideration for deployment on systems at LANL and Sandia.