KDD 1999 generation faults: a review and analysis. Issue 3 (2nd October 2018)

• Record Type: Journal Article
• Authors: Al Tobi, Amjad M.; Duncan, Ishbel
• Is Part Of: Journal of cyber security technology. Volume 2:Issue 3/4(2018)
• Page Start: 164
• Page End: 200
• Publication Date: 2018-10-02
• Subjects: DARPA 1998 dataset -- KDD Cup 1999 dataset -- KDD review -- KDD generation process -- KDD generation faults; Computer security -- Periodicals; Data encryption (Computer science) -- Periodicals; 005.805
• DOI: 10.1080/23742917.2018.1518061
• Languages: English
• ISSNs: 2374-2917

Abstract

ABSTRACT: DARPA 1998 was one of the first Intrusion Detection datasets that was made publicly available. The KDD 1999 dataset was derived from DARPA 1998 to be used by researchers in developing machine learning (ML), classification and clustering algorithms with a security focus. DARPA 1998 has been criticised in literature due to raised concerns of problems in the dataset. Many researchers have accused KDD 1999 of having similar concerns but insufficient published evidence has been found. In this paper, we review the KDD 1999 generation process and present new proofs of existing inconsistencies in KDD 1999. We then present the process used to link some of the KDD 1999 (TELNET ) records back to their origins in DARPA 1998 and discuss the interesting results and findings of this experiment.