Multicore job scheduling in the Worldwide LHC Computing Grid. Issue 6 (December 2015)

• Record Type: Journal Article
• Authors: Forti, A; Yzquierdo, A Pérez-Calero; Hartmann, T; Alef, M; Lahiff, A; Templon, J; Pra, S Dal; Gila, M; Skipsey, S; Acosta-Silva, C; Filipcic, A; Walker, R; Walker, C J; Traynor, D; Gadrat, S
• Is Part Of: Journal of physics. Volume 664:Issue 6(2015)
• Publication Date: 2015-12
• Subjects: Physics -- Congresses; 530.5
• DOI: 10.1088/1742-6596/664/6/062016
• Languages: English
• ISSNs: 1742-6588

Abstract

Abstract: After the successful first run of the LHC, data taking is scheduled to restart in Summer 2015 with experimental conditions leading to increased data volumes and event complexity. In order to process the data generated in such scenario and exploit the multicore architectures of current CPUs, the LHC experiments have developed parallelized software for data reconstruction and simulation. However, a good fraction of their computing effort is still expected to be executed as single-core tasks. Therefore, jobs with diverse resources requirements will be distributed across the Worldwide LHC Computing Grid (WLCG), making workload scheduling a complex problem in itself. In response to this challenge, the WLCG Multicore Deployment Task Force has been created in order to coordinate the joint effort from experiments and WLCG sites. The main objective is to ensure the convergence of approaches from the different LHC Virtual Organizations (VOs) to make the best use of the shared resources in order to satisfy their new computing needs, minimizing any inefficiency originated from the scheduling mechanisms, and without imposing unnecessary complexities in the way sites manage their resources. This paper describes the activities and progress of the Task Force related to the aforementioned topics, including experiences from key sites on how to best use different batch system technologies, the evolution of workload submission tools by the experiments and the knowledge gained from scale tests of the different proposed job submission strategies.