Availability modelling and assurance for a big data computing. (18th February 2019)

• Record Type: Journal Article
• Authors: Ke, Zuqiang; Park, Nohpill
• Is Part Of: International journal of big data intelligence. Volume 6:Number 2(2019)
• Page Start: 113
• Page End: 128
• Publication Date: 2019-02-18
• Subjects: availability -- map-reduce computing -- queueing model; Big data -- Periodicals; 005.705
• Languages: English
• ISSNs: 2053-1389

Abstract

This paper proposes a new analytical model to evaluate the availability of a big data computing, namely, map-reduce computing on a Hadoop platform. Map-reduce computing is represented by a queueing model in this work in order to trace flow of tasks (either map or reduce) of their arrivals and exits in the course of computation. The objective of the model is to evaluate the probability for a map-reduce computation to be available at an instance of time, referred to as availability. The set of variables taken into account in this model lists the number of map and reduce tasks, the number of servers (or referred to as nodes in this paper) engaged, along with a few constants such as task arrival/exit rates and node failure/repair rates. The proposed model provides a comprehensive yet fundamental basis to assure and ultimately optimise the design of map-reduce computing in terms of availability with reference to its performance in a simultaneous manner. Parametric simulations have been conducted and demonstrated the efficacy of the proposed model in assessing the availability and the cost.