An evaluation environment for high-performance computing combining supercomputing and cloud. (17th March 2023)

• Record Type: Journal Article
• Authors: Gotoh, Yusuke; Kotani, Toshihiro
• Is Part Of: International journal of grid and utility computing. Volume 14:Number 1(2023)
• Page Start: 29
• Page End: 36
• Publication Date: 2023-03-17
• Subjects: cloud service -- high-performance computing -- processing time -- supercomputer; Electronic data processing -- Distributed processing -- Periodicals; Electronic commerce -- Management -- Computer programs -- Periodicals; 004.605
• Languages: English
• ISSNs: 1741-847X

Abstract

In recent computer environments that use a wide variety of data, users need a system that can efficiently process data according to the scale and type of data. There are two types of typical computing environments: supercomputers and cloud systems. Cloud services can provide computer resources based on the scale of the computer environment desired by users. On the other hand, in conventional large-scale computer environment that only consists of CPUs or GPUs, the processing time greatly increases according to the scale of the calculation processing. Based on the characteristics of the supercomputer and the cloud system installed in the Hokkaido University Information Initiative Center, Japan, we aim to construct a high-performance computing environment by linking the two types of systems. In this paper, we propose an evaluation environment for high-performance computing combining supercomputing and cloud systems. Our proposed system can construct a high-performance computing environment based on the scale of the computing process by the cooperation of the supercomputing and cloud systems with short physical distance and short network distance. In our evaluation of deep reinforcement learning using our proposed system, we confirmed that computer resources can be effectively used by allocating suitable processing for the supercomputer and the cloud according to the usage situations of the CPU, the GPU, and the memory.