QLLog: A log anomaly detection method based on Q-learning algorithm. Issue 3 (May 2021)

• Record Type: Journal Article
• Authors: Duan, Xiaoyu; Ying, Shi; Yuan, Wanli; Cheng, Hailong; Yin, Xiang
• Is Part Of: Information processing & management. Volume 58:Issue 3(2021)
• Publication Date: 2021-05
• Subjects: Log anomaly detection -- Q-learning -- Reinforcement learning -- Data analysis; Information storage and retrieval systems -- Periodicals; Information science -- Periodicals; Systèmes d'information -- Périodiques; Sciences de l'information -- Périodiques; Information science; Information storage and retrieval systems; Periodicals; 658.4038
• DOI: 10.1016/j.ipm.2021.102540
• Languages: English
• ISSNs: 0306-4573

Abstract

Abstract: Most of the existing log anomaly detection methods suffer from scalability and numerous false positives. Besides, they cannot rank the severity level of abnormal events. This paper proposes a log anomaly detection based on Q-learning, namely QLLog, which can detect multiple types of system anomalies and rank the severity level of abnormal events. We first build a mathematical model of log anomaly detection, proving that log anomaly detection is a sequential decision problem. Second, we use the Q-learning algorithm to build the core of the anomaly detection model. This allows QLLog to automatically learn directed acyclic graph log patterns from normal execution and adjust the training model according to the reward value. Then, QLLog combines the advantages of the Q-learning algorithm and the specially designed rules to detect anomalies when log patterns deviate from the model trained from log data under normal execution. Besides, we provide a feedback mechanism and build an abnormal level table. Therefore, QLLog can adapt to new log states and log patterns. Experiments on real datasets show that the method can quickly and effectively detect system anomalies. Compared with the state of the art, QLLog can detect numerous real problems with high accuracy 95%, and its scalability outperforms other existing log-based anomaly detection methods. Highlights: For all we know, this paper is the first successful application of the Q-learning algorithm in the field of log anomaly detection and has achieved good detection results. QLLog can detect multiple types of log anomalies to reduce the false negative rate. QLLog provides a feedback mechanism to update the detection model and the abnormal level of abnormal logs. We summarize the existing log anomaly detection methods, compare and analyze the advantages and disadvantages of them. The experimental result proves the superiority of QLLog.