The impact factors on the competence of big data processing. Issue 3 (4th March 2022)

• Record Type: Journal Article
• Authors: Li, Wei; Guo, William W.; Li, Michael
• Is Part Of: International journal of computers and applications. Volume 44:Issue 3(2022)
• Page Start: 201
• Page End: 218
• Publication Date: 2022-03-04
• Subjects: Volunteer computing -- big data -- impact factor -- scalability -- MapReduce; Computers -- Periodicals; Computer software -- Periodicals; Computer networks -- Periodicals; Multimedia systems -- Periodicals; Internet -- Periodicals; World Wide Web -- Periodicals; Minicomputers -- Periodicals; Microcomputers -- Periodicals; 004.05
• DOI: 10.1080/1206212X.2020.1719623
• Languages: English
• ISSNs: 1206-212X

Abstract

Abstract : A major challenge of using volunteer computing (VC) for big data problems is the opportunistic environment. In such a dynamic, unreliable, and heterogeneous environment, there exist many uncertain factors that may impair the competence of big data processing. This paper explores these factors and their impact, aiming at exposing the original impaired performance that the undedicated environment of VC can achieve under the impact. Our investigation on this issue is four-fold. First, we define a number of impact factors to represent the opportunistic features of VC environment. Second, we model a distributed hash table-based MapReduce approach to process big data. Third, we proposes an ideal computing environment and then inject impact factors into the running MapReduce to quantitatively evaluate how the performance is impaired. Finally, we analyze the evaluation results to figure out the cause of impact and predict optimization potentials. The developers, who plan to construct MapReduce frameworks by using commodity computers or voluntary cycles on the public internet, will benefit from the evaluation results when considering performance requirements.