Medical data quality assessment: On the development of an automated framework for medical data curation. (April 2019)

• Record Type: Journal Article
• Authors: Pezoulas, Vasileios C.; Kourou, Konstantina D.; Kalatzis, Fanis; Exarchos, Themis P.; Venetsanopoulou, Aliki; Zampeli, Evi; Gandolfo, Saviana; Skopouli, Fotini; De Vita, Salvatore; Tzioufas, Athanasios G.; Fotiadis, Dimitrios I.
• Is Part Of: Computers in biology and medicine. Volume 107(2019)
• Page Start: 270
• Page End: 283
• Publication Date: 2019-04
• Subjects: Big data -- Data quality -- Data quality assessment -- Data curation -- Data standardization; Medicine -- Data processing -- Periodicals; Biology -- Data processing -- Periodicals; 610.285
• DOI: 10.1016/j.compbiomed.2019.03.001
• Languages: English
• ISSNs: 0010-4825

Abstract

Abstract: Data quality assessment has gained attention in the recent years since more and more companies and medical centers are highlighting the importance of an automated framework to effectively manage the quality of their big data. Data cleaning, also known as data curation, lies in the heart of the data quality assessment and is a key aspect prior to the development of any data analytics services. In this work, we present the objectives, functionalities and methodological advances of an automated framework for data curation from a medical perspective. The steps towards the development of a system for data quality assessment are first described along with multidisciplinary data quality measures. A three-layer architecture which realizes these steps is then presented. Emphasis is given on the detection and tracking of inconsistencies, missing values, outliers, and similarities, as well as, on data standardization to finally enable data harmonization. A case study is conducted in order to demonstrate the applicability and reliability of the proposed framework on two well-established cohorts with clinical data related to the primary Sjögren's Syndrome (pSS). Our results confirm the validity of the proposed framework towards the automated and fast identification of outliers, inconsistencies, and highly-correlated and duplicated terms, as well as, the successful matching of more than 85% of the pSS-related medical terms in both cohorts, yielding more accurate, relevant, and consistent clinical data. Graphical abstract: Image 1