Bursty event detection from microblog: a distributed and incremental approach. (20th September 2015)

• Record Type: Journal Article
• Authors: Li, Jianxin; Wen, Jianfeng; Tai, Zhenying; Zhang, Richong; Yu, Weiren
• Other Names: Frincu Mark guestEditor.; Bósa Károly guestEditor.; Rong Chunming guestEditor.; Liu Lu guestEditor.; Chen Guolong guestEditor.
• Is Part Of: Concurrency and computation. Volume 28:Number 11(2016)
• Page Start: 3115
• Page End: 3130
• Publication Date: 2015-09-20
• Subjects: social network -- event detection -- temporal topic model -- topic drifting; Parallel processing (Electronic computers) -- Periodicals; Parallel computers -- Periodicals; 004.35
• DOI: 10.1002/cpe.3657
• Languages: English
• ISSNs: 1532-0626

Abstract

Summary: As a new form of social media, microblogs (e.g., Twitter and Weibo) are playing an important role in people's daily life. With the rise in popularity and size of microblogs, there is a need for distributed approaches that can detect bursty event with low latency from the short‐text data stream. In this paper, we propose a distributed and incremental temporal topic model for microblogs called Bursty Event dEtection (BEE+). BEE+ is able to detect bursty events from short‐text dataset and model the temporal information. And BEE+ processes the post‐stream incrementally to track the topic drifting of events over time. Therefore, the latent semantic indices are preserved from one time period to the next. In order to achieve real‐time processing, we design a distributed execution framework based on Spark engine. To verify its ability to detect bursty event, we conduct experiments on a Weibo dataset of 6, 360, 125 posts. The results show that BEE+ can outperform the baselines for detecting the meaningful bursty events and track the topic drifting. Copyright © 2015 John Wiley & Sons, Ltd.