Variance and density-based anomaly identification and ranking for evolving data streams. (1st January 2014)

• Record Type: Journal Article
• Authors: Durga, Toshniwal
• Is Part Of: International journal of computational intelligence studies. Volume 3:Number 2/3(2014)
• Page Start: 251
• Page End: 274
• Publication Date: 2014-01-01
• Subjects: anomaly detection -- evolving data streams -- dataset of varying density -- variance-based clustering; Computational intelligence -- Periodicals; 006.305
• Languages: English
• ISSNs: 1755-4985

Abstract

Data stream mining is emerging as an important research area in the recent times. This is due the fact that streaming data gets generated from many applications. Stream mining is very challenging because streaming data cannot be scanned multiple times and also new concepts may keep evolving in the data. Finding anomalies from data streams is of great significance in many applications. Most of the existing anomaly detection techniques suffer from many limitations. They are commonly applicable to static data of uniform densities. But the majority of real world data is of varying densities. In the present work, we propose a method which identifies anomalies based on variance and data density and also assigns ranks to anomalies. Initially, variance-based clustering is done to find the candidate anomalies. After finding the candidate anomalies, we assign ranks to them based on the densities of the clusters. Furthermore, to reduce the effect of irrelevant (noisy) attributes during anomaly detection, the proposed method assigns weights to attributes depending upon their respective relevance. Keeping in view the challenges of streaming data, the proposed method is incremental and adaptive to evolution of new concept in the data. Experimental results on both synthetic and real world datasets show that the proposed method outperforms other existing methods.