One-pass online learning: A local approach. (March 2016)

• Record Type: Journal Article
• Authors: Zhou, Zhaoze; Zheng, Wei-Shi; Hu, Jian-Fang; Xu, Yong; You, Jane
• Is Part Of: Pattern recognition. Volume 51(2016:Mar.)
• Page Start: 346
• Page End: 357
• Publication Date: 2016-03
• Subjects: One-pass online learning -- Local modeling -- Classification; Pattern perception -- Periodicals; Perception des structures -- Périodiques; Patroonherkenning; 006.4
• DOI: 10.1016/j.patcog.2015.09.003
• Languages: English
• ISSNs: 0031-3203

Abstract

Abstract: Online learning is very important for processing sequential data and helps alleviate the computation burden on large scale data as well. Especially, one-pass online learning is to predict a new coming sample׳s label and update the model based on the prediction, where each coming sample is used only once and never stored. So far, existing one-pass online learning methods are globally modeled and do not take the local structure of the data distribution into consideration, which is a significant factor of handling the nonlinear data separation case. In this work, we propose a local online learning (LOL) method, a multiple hyperplane Passive Aggressive algorithm integrated with online clustering, so that all local hyperplanes are learned jointly and working cooperatively. This is achieved by formulating a common component as information traffic among multiple hyperplanes in LOL. A joint optimization algorithm is proposed and theoretical analysis on the cumulative error is also provided. Extensive experiments on 11 datasets show that LOL can learn a nonlinear decision boundary, overall achieving notably better performance without using any kernel modeling and second order modeling. Abstract : Highlights: Propose an one-pass local online learning algorithm (LOL). LOL learns multiple hyperplanes jointly. LOL makes non-linear online learning more effective and accurate. Provide theoretical analysis on the cumulative error of LOL. Experimentally show the effectiveness of the proposed method.