SOINN+, a Self-Organizing Incremental Neural Network for Unsupervised Learning from Noisy Data Streams. (1st April 2020)

• Record Type: Journal Article
• Authors: Wiwatcharakoses, Chayut; Berrar, Daniel
• Is Part Of: Expert systems with applications. Volume 143(2020)
• Publication Date: 2020-04-01
• Subjects: Continuous learning -- Catastrophic forgetting -- Self-organizing incremental neural network -- Non-stationary data -- Data stream -- Concept drift; Expert systems (Computer science) -- Periodicals; Systèmes experts (Informatique) -- Périodiques; Electronic journals; 006.33
• DOI: 10.1016/j.eswa.2019.113069
• Languages: English
• ISSNs: 0957-4174

Abstract

Highlights: We present a new self-organizing incremental neural network: SOINN+. SOINN+ is for unsupervised, continuous learning. SOINN+ can detect clusters in noisy, evolving data streams. SOINN+ mitigates catastrophic forgetting. Abstract: The goal of continuous learning is to acquire and fine-tune knowledge incrementally without erasing already existing knowledge. How to mitigate this erasure, known as catastrophic forgetting, is a grand challenge for machine learning, specifically when systems are trained on evolving data streams. Self-organizing incremental neural networks (SOINN) are a class of neural networks that are designed for continuous learning from non-stationary data. Here, we propose a novel method, SOINN+, which is fundamentally different from the previous generations of SOINN with respect to how "forgetting" is modeled. To achieve a more graceful forgetting, we developed three new concepts: idle time, trustworthiness, and unutility of a node. SOINN+ learns a topology-preserving mapping of the input data to a network structure, which reveals clusters of arbitrary shapes in streams of noisy data. Our experiments with synthetic and real-world data sets showed that SOINN+ can maintain discovered structures even under sudden and recurring concept drifts.