Choice of parallelism: multi-GPU driven pipeline for huge academic backbone network. Issue 6 (2nd November 2021)
- Record Type:
- Journal Article
- Title:
- Choice of parallelism: multi-GPU driven pipeline for huge academic backbone network. Issue 6 (2nd November 2021)
- Main Title:
- Choice of parallelism: multi-GPU driven pipeline for huge academic backbone network
- Authors:
- Ando, Ruo
Kadobayashi, Youki
Takakura, Hiroki - Abstract:
- Abstract : Science Information Network (SINET) is a Japanese academic backbone network for more than 800 research institutions and universities. In this paper, we present a multi-GPU-driven pipeline for handling huge session data of SINET. Our pipeline consists of ELK stack, multi-GPU server, and Splunk. A multi-GPU server is responsible for two procedures: discrimination and histogramming. Discrimination is dividing session data into ingoing/outgoing with subnet mask calculation and network address matching. Histogramming is grouping ingoing/outgoing session data into bins with map-reduce. In our architecture, we use GPU for the acceleration of ingress/egress discrimination of session data. Also, we use a tiling design pattern for building a two-stage map-reduce of CPU and GPU. Our multi-GPU-driven pipeline has succeeded in processing huge workloads of about 1.2–1.6 billion session streams (500–650 GB) within 24 hours. GRAPHICAL ABSTRACT: UF0001
- Is Part Of:
- International journal of parallel, emergent and distributed systems. Volume 36:Issue 6(2021)
- Journal:
- International journal of parallel, emergent and distributed systems
- Issue:
- Volume 36:Issue 6(2021)
- Issue Display:
- Volume 36, Issue 6 (2021)
- Year:
- 2021
- Volume:
- 36
- Issue:
- 6
- Issue Sort Value:
- 2021-0036-0006-0000
- Page Start:
- 609
- Page End:
- 622
- Publication Date:
- 2021-11-02
- Subjects:
- Multi-GPU -- streaming data pipeline -- massive session data -- map reduce -- tiling pattern
Parallel computers -- Periodicals
Electronic data processing -- Distributed processing -- Periodicals
Computer algorithms -- Periodicals
004.35 - Journal URLs:
- http://www.tandfonline.com/toc/gpaa20/current ↗
http://www.tandfonline.com/ ↗ - DOI:
- 10.1080/17445760.2021.1941009 ↗
- Languages:
- English
- ISSNs:
- 1744-5760
- Deposit Type:
- Legaldeposit
- View Content:
- Available online (eLD content is only available in our Reading Rooms) ↗
- Physical Locations:
- British Library DSC - 4542.441300
British Library DSC - BLDSS-3PM
British Library STI - ELD Digital store - Ingest File:
- 19396.xml