Incorporating change detection in the monitoring phase of adaptive query processing. Issue 1 (December 2016)

• Record Type: Journal Article
• Authors: Tsamoura, Efthymia; Gounaris, Anastasios; Manolopoulos, Yannis
• Is Part Of: Journal of internet services and applications. Volume 7:Issue 1(2016)
• Page Start: 1
• Page End: 18
• Publication Date: 2016-12
• Subjects: Adaptive query processing -- Stream data processing -- Stream change detection; TCP/IP (Computer network protocol) -- Periodicals; Internet -- Periodicals; Internet -- Security measures -- Periodicals; 004.678
• DOI: 10.1186/s13174-016-0049-5
• Languages: English
• ISSNs: 1867-4828

Abstract

Abstract Recent Big Data research typically emphasizes on the need to address the challenges stemming from the volume, velocity, variety and veracity aspects. However, another cross-cutting property of Big Data is volatility. In database technology, volatility is addressed with the help of adaptive query processing (AQP), which has become the dominant paradigm for executing queries in dynamic and/or streaming environments. As the characteristics of the runtime environment may vary significantly along time, AQP techniques employ a three-phase adaptivity loop to process the input queries, comprising feedback collection, analysis and re-optimization. In the monitoring phase, the standard approach is to collect feedback in a fixed-size sliding window. However, several problems arise when the techniques adopt a fixed-size sliding window for maintaining runtime collected feedback. In this work, we tackle this limitation and we propose a novel monitoring phase, which assesses the collected feedback rendering an AQP technique capable of taking more informed decisions during the subsequent phases. The proposed approach is non-intrusive to the state-of-the-art adaptivity loop and can adopt any state-of-the-art online change detection algorithm through its plug-and-play abstraction. Another contribution of this work is a novel algorithm for detecting changes in a filter's drop probability, calledβ -CUSUM. The potential of the novel monitoring phase and ofβ -CUSUM is experimentally evaluated using both real-world and synthetic datasets.