A platform for big data analytics on distributed scale-out storage system. (2015)

• Record Type: Journal Article
• Authors: Aye, Kyar Nyo; Thein, Thandar
• Is Part Of: International journal of big data intelligence. Volume 2:Number 2(2015)
• Page Start: 127
• Page End: 141
• Publication Date: 2015
• Subjects: big data analytics -- big data platforms -- Hadoop MapReduce -- Gluster file system -- Apache Pig -- Apache Hive -- Jaql -- distributed storage systems -- scale-out storage systems -- metadata storage -- big data storage -- big data processing -- performance evaluation -- scalability -- fault tolerance -- query response time; Big data -- Periodicals; 005.705
• Languages: English
• ISSNs: 2053-1389

Abstract

Big data analytics is the process of examining large amounts of data of a variety of types to uncover hidden patterns, unknown correlations and other useful information. Hadoop-based platform emerges to deal with big data. In Hadoop NameNode is used to store metadata in a single system's memory, which is a performance bottleneck for scale-out. Gluster file system has no performance bottlenecks related to metadata. To achieve massive performance, scalability and fault tolerance for big data analytics, a big data platform is proposed. The proposed big data platform consists of big data storage and big data processing. The Hadoop big data platform and the proposed big data platform are implemented on commodity Linux virtual machines clusters and performance evaluations are conducted. According to the evaluation analysis, the proposed big data platform provides better scalability, fault tolerance, and faster query response time than the Hadoop platform.