Potential threats mining methods based on correlation analysis of multi‐type logs. Issue 5 (1st September 2018)

• Record Type: Journal Article
• Authors: Qin, Tao; Gao, Yuli; Wei, Lingyan; Liu, Zhaoli; Wang, Chenxu
• Is Part Of: IET networks. Volume 7:Issue 5(2018)
• Page Start: 299
• Page End: 305
• Publication Date: 2018-09-01
• Subjects: computer network security -- data mining -- regression analysis -- feature extraction -- operating systems (computers) -- computational complexity; correlation analysis -- potential attacks -- heterogeneous features -- feature normalisation -- anomaly detection platform -- massive logs -- multitype logs -- log analysis -- log events -- operating systems -- potential threats mining methods -- network level -- feature extraction -- campus network -- Xi'an Jiaotong University -- computational complexity -- Flume/HDFS/Spark; Computer network architectures -- Periodicals; Computer network protocols -- Periodicals; Information networks -- Periodicals; Telecommunication systems -- Periodicals; 004.605
• DOI: 10.1049/iet-net.2017.0188
• Languages: English
• ISSNs: 2047-4954

Abstract

Abstract : Log analysis is an efficiency way to detect threats by scrutinizing the events recorded by the operating systems and devices. However, it is more and more difficult to discover threats accurately due to the massive amount of logs and their various formats. Focusing on this problem, the authors propose a method for potential threats mining based on the correlation analysis of multi‐type logs. Firstly, they extract 12 features, including behavior‐related, attribute‐related and measurable features, from multi‐type logs based on the characteristics of known and potential attacks. They also propose normalization method to deal with these heterogeneous features. Secondly, focusing on solving the problem that analyzing a single type of log can only detect some specific attacks, they employ the logistic regression model to perform correlation analysis on multi‐type logs. Finally, they construct an anomaly detection platform integrated with parallel processing mechanism to process the massive records. The experimental results based on logs collected show that the proposed method has high detection accuracy and low computational complexity, which can be applied to mine potential threats and abnormal users from the massive logs in an actual network environment.