Merging weighted SVMs for parallel incremental learning. (April 2018)

• Record Type: Journal Article
• Authors: Zhu, Lei; Ikeda, Kazushi; Pang, Shaoning; Ban, Tao; Sarrafzadeh, Abdolhossein
• Is Part Of: Neural networks. Volume 100(2018)
• Page Start: 25
• Page End: 38
• Publication Date: 2018-04
• Subjects: Incremental learning -- Parallel learning -- Parallel incremental learning -- Knowledge merging -- Extreme support vector machine (ESVM) -- Weighted ESVM (wESVM); Neural computers -- Periodicals; Neural networks (Computer science) -- Periodicals; Neural networks (Neurobiology) -- Periodicals; Nervous System -- Periodicals; Ordinateurs neuronaux -- Périodiques; Réseaux neuronaux (Informatique) -- Périodiques; Réseaux neuronaux (Neurobiologie) -- Périodiques; Neural computers; Neural networks (Computer science); Neural networks (Neurobiology); Periodicals; 006.32
• DOI: 10.1016/j.neunet.2018.01.001
• Languages: English
• ISSNs: 0893-6080

Abstract

Abstract: Parallel incremental learning is an effective approach for rapidly processing large scale data streams, where parallel and incremental learning are often treated as two separate problems and solved one after another. Incremental learning can be implemented by merging knowledge from incoming data and parallel learning can be performed by merging knowledge from simultaneous learners. We propose to simultaneously solve the two learning problems with a single process of knowledge merging, and we propose parallel incremental wESVM (weighted Extreme Support Vector Machine) to do so. Here, wESVM is reformulated such that knowledge from subsets of training data can be merged via simple matrix addition. As such, the proposed algorithm is able to conduct parallel incremental learning by merging knowledge over data slices arriving at each incremental stage. Both theoretical and experimental studies show the equivalence of the proposed algorithm to batch wESVM in terms of learning effectiveness. In particular, the algorithm demonstrates desired scalability and clear speed advantages to batch retraining.