Data transformation as a means towards dynamic data storage and polyglot persistence. Issue 4 (4th May 2017)
- Record Type:
- Journal Article
- Title:
- Data transformation as a means towards dynamic data storage and polyglot persistence. Issue 4 (4th May 2017)
- Main Title:
- Data transformation as a means towards dynamic data storage and polyglot persistence
- Authors:
- Vanhove, Thomas
Sebrechts, Merlijn
Van Seghbroeck, Gregory
Wauters, Tim
Volckaert, Bruno
De Turck, Filip
Diao, Yixin
Zincir‐Heywood, A. Nur - Abstract:
- Summary: Legacy applications have been built around the concept of storing their data in one relational data store. However, with the current differentiation in data store technologies as a consequence of the NoSQL paradigm, new and possibly more performant storage solutions are available to all applications. The concept of dynamic storage makes sure that application data are always stored in the most optimal data store at a given time to increase application performance. Additionally, polyglot persistence aims to push this performance even further by storing each different data type of an application in the data store technology best suited for it. To get legacy applications into dynamic storage and polyglot persistence, schema and data transformations between data store technologies are needed. This usually infers application redesigns as well to support the new data stores. This paper proposes such a transformation approach through a canonical model. It is based on the Lambda architecture to ensure no application downtime is needed during the transformation process, and after the transformation, the application can continue to query in the original query language, thus requiring no application code changes. Abstract : Big data sets are often stored in suboptimal data store technologies, but data transformations is are costly operations. To that end, this article introduces a transformation algorithm through a centralized canonical model without the need for data store orSummary: Legacy applications have been built around the concept of storing their data in one relational data store. However, with the current differentiation in data store technologies as a consequence of the NoSQL paradigm, new and possibly more performant storage solutions are available to all applications. The concept of dynamic storage makes sure that application data are always stored in the most optimal data store at a given time to increase application performance. Additionally, polyglot persistence aims to push this performance even further by storing each different data type of an application in the data store technology best suited for it. To get legacy applications into dynamic storage and polyglot persistence, schema and data transformations between data store technologies are needed. This usually infers application redesigns as well to support the new data stores. This paper proposes such a transformation approach through a canonical model. It is based on the Lambda architecture to ensure no application downtime is needed during the transformation process, and after the transformation, the application can continue to query in the original query language, thus requiring no application code changes. Abstract : Big data sets are often stored in suboptimal data store technologies, but data transformations is are costly operations. To that end, this article introduces a transformation algorithm through a centralized canonical model without the need for data store or application downtime. The transformation optimizes the representation of the data for each specific technology ultimately aiming at decreasing query latency for applications. Results show data sets can be correctly transformed and optimized within a reasonable time frame effectively increasing application performance. … (more)
- Is Part Of:
- International journal of network management. Volume 27:Issue 4(2017)
- Journal:
- International journal of network management
- Issue:
- Volume 27:Issue 4(2017)
- Issue Display:
- Volume 27, Issue 4 (2017)
- Year:
- 2017
- Volume:
- 27
- Issue:
- 4
- Issue Sort Value:
- 2017-0027-0004-0000
- Page Start:
- n/a
- Page End:
- n/a
- Publication Date:
- 2017-05-04
- Subjects:
- big data -- data transformation -- dynamic data storage -- lambda architecture -- polyglot persistence
Computer networks -- Management -- Periodicals
004.6 - Journal URLs:
- http://onlinelibrary.wiley.com/journal/10.1002/(ISSN)1099-1190 ↗
http://onlinelibrary.wiley.com/ ↗ - DOI:
- 10.1002/nem.1976 ↗
- Languages:
- English
- ISSNs:
- 1055-7148
- Deposit Type:
- Legaldeposit
- View Content:
- Available online (eLD content is only available in our Reading Rooms) ↗
- Physical Locations:
- British Library DSC - 4542.373300
British Library DSC - BLDSS-3PM
British Library STI - ELD Digital store - Ingest File:
- 2895.xml