D2IA: User-defined interval analytics on distributed streams. Issue 104 (February 2022)

• Record Type: Journal Article
• Authors: Awad, Ahmed; Tommasini, Riccardo; Langhi, Samuele; Kamel, Mahmoud; Della Valle, Emanuele; Sakr, Sherif
• Is Part Of: Information systems. Issue 104(2022)
• Publication Date: 2022-02
• Subjects: Big Stream Processing -- Complex Event Processing -- User-defined event intervals; Database management -- Periodicals; Electronic data processing -- Periodicals; Bases de données -- Gestion -- Périodiques; Informatique -- Périodiques; Database management; Electronic data processing; Periodicals; 005.7
• DOI: 10.1016/j.is.2020.101679
• Languages: English
• ISSNs: 0306-4379

Abstract

Abstract: Nowadays, modern Big Stream Processing Solutions (e.g. Spark, Flink ) are working towards being the ultimate framework for streaming analytics. In order to achieve this goal, they started to offer extensions of SQL that incorporate stream-oriented primitives such as windowing and Complex Event Processing (CEP). The former enables stateful computation on infinite sequences of data items while the latter focuses on the detection of events pattern. In most of the cases, data items and events are considered instantaneous, i.e., they are single time points in a discrete temporal domain. Nevertheless, a point-based time semantics does not satisfy the requirements of a number of use-cases. For instance, it is not possible to detect the interval during which the temperature increases until the temperature begins to decrease, nor for all the relations this interval subsumes. To tackle this challenge, we present D 2 IA ; a set of novel abstract operators to define analytics on user-defined event intervals based on raw events and to efficiently reason about temporal relationships between intervals and/or point events. We realize the implementation of the concepts of D 2 IA on top of Flink, a distributed stream processing engine for big data. Highlights: A family of operators to generate interval events from instantaneous events. Operators are defined as DSL to expressively generate data-driven intervals. Operators fill a gap in the expressiveness of large-scale stream processing engines. Two alternative implementations based on Apache Flink and Esper. A systematic perfromance evaluation using the linear road benchmark.