Topology-aware job mapping. (January 2018)

• Record Type: Journal Article
• Authors: Georgiou, Yiannis; Jeannot, Emmanuel; Mercier, Guillaume; Villiermet, Adèle
• Other Names: Dongarra Jack guest-editor.; Tourancheau Bernard guest-editor.
• Is Part Of: International journal of high performance computing applications. Volume 32:Number 1(2018)
• Page Start: 14
• Page End: 27
• Publication Date: 2018-01
• Subjects: Algorithm -- high-performance computing -- job scheduling -- process placement -- topology; High performance computing -- Periodicals; Supercomputers -- Periodicals; 004.1105
• DOI: 10.1177/1094342017727061
• Languages: English
• ISSNs: 1094-3420

Abstract

A Resource and Job Management System (RJMS) is a crucial system software part of the HPC stack. It is responsible for efficiently delivering computing power to applications in supercomputing environments. Its main intelligence relies on resource selection techniques to find the most adapted resources to schedule the users' jobs. This article introduces a new method that takes into account the topology of the machine and the application characteristics to determine the best choice among the available nodes of the platform, based upon the network topology and taking into account the application communication pattern. To validate our approach, we integrate this algorithm as a plugin for Simple Linux Utility for Resource Management (SLURM), a well-known and widespread RJMS. We assess our plugin with different optimization schemes by comparing with the default topology-aware Slurm algorithm, using both emulation and simulation of a large-scale platform and by carrying out experiments in a real cluster. We show that transparently taking into account a job communication pattern and the topology allows for relevant performance gains.