Selective ensemble-based online adaptive deep neural networks for streaming data with concept drift. (October 2021)

• Record Type: Journal Article
• Authors: Guo, Husheng; Zhang, Shuai; Wang, Wenjian
• Is Part Of: Neural networks. Volume 142(2021)
• Page Start: 437
• Page End: 456
• Publication Date: 2021-10
• Subjects: Concept drift -- Selective ensemble -- Deep neural networks -- Online learning -- Adaptive method; Neural computers -- Periodicals; Neural networks (Computer science) -- Periodicals; Neural networks (Neurobiology) -- Periodicals; Nervous System -- Periodicals; Ordinateurs neuronaux -- Périodiques; Réseaux neuronaux (Informatique) -- Périodiques; Réseaux neuronaux (Neurobiologie) -- Périodiques; Neural computers; Neural networks (Computer science); Neural networks (Neurobiology); Periodicals; 006.32
• DOI: 10.1016/j.neunet.2021.06.027
• Languages: English
• ISSNs: 0893-6080

Abstract

Abstract: Concept drift is an important issue in the field of streaming data mining. However, how to maintain real-time model convergence in a dynamic environment is an important and difficult problem. In addition, the current methods have limited ability to deal with the problem of streaming data classification for complex nonlinear problems. To solve these problems, a selective ensemble-based online adaptive deep neural network (SEOA) is proposed to address concept drift. First, the adaptive depth unit is constructed by combining shallow features with deep features and adaptively controls the information flow in the neural network according to changes in streaming data at adjacent moments, which improves the convergence of the online deep learning model. Then, the adaptive depth units of different layers are regarded as base classifiers for ensemble and weighted dynamically according to the loss of each classifier. In addition, a dynamic selection of base classifiers is adopted according to the fluctuation of the streaming data to achieve a balance between stability and adaptability. The experimental results show that the SEOA can effectively contend with different types of concept drift and has good robustness and generalization. Highlights: Adaptive combination of shallow feature and deep feature to improve model convergence. Dynamic selection of basic classifiers with different properties thus achieving balance between stability and adaptability. The problem of gradient vanishing and feature reuse diminishing is solved.