A Conceptual Approach for Supporting Traffic Data Wrangling Tasks. (3rd November 2018)

• Record Type: Journal Article
• Authors: Sampaio, Sandra; Aljubairah, Mashael; Permana, Hapsoro Adi; Sampaio, Pedro
• Is Part Of: Computer journal. Volume 62:Number 3(2019)
• Page Start: 461
• Page End: 480
• Publication Date: 2018-11-03
• Subjects: data wrangling -- data transformation and quality -- conceptual wrangling approaches; Computers -- Periodicals; 005.1
• DOI: 10.1093/comjnl/bxy113
• Languages: English
• ISSNs: 0010-4620

Abstract

Abstract: Data wrangling (DW) is the subject of growing interest given its potential to improve data quality. DW applies interactive and iterative data profiling, cleaning, transformation, integration and visualization operations to improve the quality of data. Several domain-independent DW tools have been developed to tackle data quality issues across domains. Using generic DW tools requires a time-consuming and costly DW process, often involving advanced IT knowledge beyond the skills set of traffic analysts. In this paper, we propose a conceptual approach to DW for traffic data by creating a domain-specific language for specifying traffic DW tasks and an abstract set of wrangling operators that serve as the target conceptual construct for mapping domain-specific wrangling tasks. The conceptual approach discussed in this paper is tool-independent and platform agnostic and can be mapped into specific implementations of DW functions available in existing scripting languages and tools such as R, Python, Trifacta. Our aim is to enable a typical traffic analyst without expert Data Science knowledge to be able to perform basic DW tasks relevant to his domain.