Network traffic performance analysis from passive measurements using gradient boosting machine learning. (25th October 2021)

• Record Type: Journal Article
• Authors: Syal, Astha; Lazar, Alina; Kim, Jinoh; Sim, Alex; Wu, Kesheng
• Is Part Of: International journal of big data intelligence. Volume 8:Number 1(2021)
• Page Start: 13
• Page End: 30
• Publication Date: 2021-10-25
• Subjects: network traffic -- TCP performance -- UMAP -- classification -- Tstat; Big data -- Periodicals; 005.705
• Languages: English
• ISSNs: 2053-1389

Abstract

Effective monitoring and analysis of network traffic are vital for scientific computing, since scientific applications often require moving massive data from one site to another. A body of statistical and machine learning techniques have been introduced for network traffic monitoring and analysis, but this is considered a highly challenging task due to several reasons, such as unavailability of label information, complication of real-time analysis, and generalisation property of machine learning models. In this paper, we present a novel method to identify continuous time windows of low throughput for the purpose of network performance analysis and anomaly detection, in order to facilitate data transfers for high-performance scientific computing. The presented method is based on supervised learning techniques with an adaptive labelling function that automatically determines if the time window is whether 'slow' or 'normal'. The presented method is validated on real large datasets collected from several data transfer nodes (DTNs) located in Science DMZ. Our experimental results show that the proposed method is able to quickly predict time windows of low performing network transfers, that require attention from network engineers and also to identify the most important features for the classification.