A comparison of two blending-based ensemble techniques for network anomaly detection in Spark distributed environment. (11th September 2020)

• Record Type: Journal Article
• Authors: Kaur, Gagandeep; Jain, Meenal
• Is Part Of: International journal of ad hoc and ubiquitous computing. Volume 35:Number 2(2020)
• Page Start: 71
• Page End: 83
• Publication Date: 2020-09-11
• Subjects: resilient distributed data structures -- Apache Spark -- clustering -- K-means -- Gaussian mixture model -- GMM -- random forest -- ensemble -- anomaly detection; Ubiquitous computing -- Periodicals; Embedded computer systems -- Periodicals; Electronic data processing -- Distributed processing -- Periodicals; Wireless communication systems -- Periodicals; Computer architecture -- Periodicals; 004.2
• Languages: English
• ISSNs: 1743-8225

Abstract

In this paper, two blending-based ensemble models, namely, logistic regression-based blending ensemble and SVM-based blending ensemble have been compared in terms of their total training time in a distributed environment and their detection accuracy rates. To handle process of concept drift two clustering algorithms have been compared for their training times in a distributed environment. Tests have been conducted on different machines by varying the number of executor cores to study time latency in a distributed Spark environment. Logistic regression-based blending ensemble with an accuracy of 93% and an accuracy of 98% using SVM-based blending ensemble was achieved. The proposed models have been evaluated using CIDDS dataset.