Discriminative structure learning of sum–product networks for data stream classification. (March 2020)

• Record Type: Journal Article
• Authors: Sun, Zhengya; Liu, Cheng-Lin; Niu, Jinghao; Zhang, Wensheng
• Is Part Of: Neural networks. Volume 123(2020)
• Page Start: 163
• Page End: 175
• Publication Date: 2020-03
• Subjects: Sum–product network -- Discriminative structure learning -- Data stream classification; Neural computers -- Periodicals; Neural networks (Computer science) -- Periodicals; Neural networks (Neurobiology) -- Periodicals; Nervous System -- Periodicals; Ordinateurs neuronaux -- Périodiques; Réseaux neuronaux (Informatique) -- Périodiques; Réseaux neuronaux (Neurobiologie) -- Périodiques; Neural computers; Neural networks (Computer science); Neural networks (Neurobiology); Periodicals; 006.32
• DOI: 10.1016/j.neunet.2019.12.002
• Languages: English
• ISSNs: 0893-6080

Abstract

Abstract: Sum–product network (SPN) is a deep probabilistic representation that allows for exact and tractable inference. There has been a trend of online SPN structure learning from massive and continuous data streams. However, online structure learning of SPNs has been introduced only for the generative settings so far. In this paper, we present an online discriminative approach for SPNs for learning both the structure and parameters. The basic idea is to keep track of informative and representative examples to capture the trend of time-changing class distributions. Specifically, by estimating the goodness of model fitting of data points and dynamically maintaining a certain amount of informative examples over time, we generate new sub-SPNs in a recursive and top-down manner. Meanwhile, an outlier-robust margin-based log-likelihood loss is applied locally to each data point and the parameters of SPN are updated continuously using most probable explanation (MPE) inference. This leads to a fast yet powerful optimization procedure and improved discrimination capability between the genuine class and rival classes. Empirical results show that the proposed approach achieves better prediction performance than the state-of-the-art online structure learner for SPNs, while promising order-of-magnitude speedup. Comparison with state-of-the-art stream classifiers further proves the superiority of our approach.