Practical implications of using non‐relational databases to store large genomic data files and novel phenotypes. (29th August 2021)

• Record Type: Journal Article
• Authors: Moreira Souza, André; Weigert, Rodrigo de Andrade Santos; Machado de Sousa, Elaine Parros; Tassoni Andrietta, Lucas; Ventura, Ricardo Vieira
• Is Part Of: Journal of animal breeding and genetics. Volume 139:Number 1(2022)
• Page Start: 100
• Page End: 112
• Publication Date: 2021-08-29
• Subjects: FASTQ -- MongoDB -- novel phenotypes -- SNP; Livestock -- Breeding -- Periodicals; Livestock -- Genetics -- Periodicals; 636.0820
• DOI: 10.1111/jbg.12644
• Languages: English
• ISSNs: 0931-2668

Abstract

Abstract: The objective of our study was to provide practical directions on the storage of genomic information and novel phenotypes (treated here as unstructured data) using a non‐relational database. The MongoDB technology was assessed for this purpose, enabling frequent data transactions involving numerous individuals under genetic evaluation. Our study investigated different genomic (Illumina Final Report, PLINK, 0125, FASTQ, and VCF formats) and phenotypic (including media files) information, using both real and simulated datasets. Advantages of our centralized database concept include the sublinear running time for queries after increasing the number of samples/markers exponentially, in addition to the comprehensive management of distinct data formats while searching for specific genomic regions. A comparison of our non‐relational and generic solution, with an existing relational approach (developed for tabular data types using 2 bits to store genotypes), showed reduced importing time to handle 50M SNPs (PLINK format) achieved by the relational schema. Our experimental results also reinforce that data conversion is a costly step required to manage genomic data into both relational and non‐relational database systems, and therefore, must be carefully treated for large applications.