Adapting bioinformatics applications for heterogeneous systems: a case study‡. (25th September 2012)

• Record Type: Journal Article
• Authors: Lanc, Irena; Bui, Peter; Thain, Douglas; Emrich, Scott; Qiu, Judy; Foster, Ian; Taylor, Ronald; Loidl, Hans‐Wolfgang; Singer, Jeremy
• Is Part Of: Concurrency and computation. Volume 26:Number 4(2014:Mar.)
• Page Start: 866
• Page End: 877
• Publication Date: 2012-09-25
• Subjects: Parallel processing (Electronic computers) -- Periodicals; Parallel computers -- Periodicals; 004.35
• DOI: 10.1002/cpe.2927
• Languages: English
• ISSNs: 1532-0626

Abstract

<abstract abstract-type="main" id="cpe2927-abs-0001"> <title>SUMMARY</title> <p id="cpe2927-para-0002">The advent of new sequencing technologies has generated extremely large amounts of information. To successfully apply bioinformatics tools to such large datasets, they need to exhibit scalability and ideally elasticity in diverse computing environments. We describe the application of previously obtained lessons to a new workflow with and without shared file storage. Because the original workflows have an intractable sequential running times on large datasets, we propose lessons and results for refactoring bioinformatics tools for elastic scaling on personal clouds. Our case studies describe the various challenges faced when constructing such a workflow, from dealing with failure detection, to managing dependencies, to handling the quirks of the underlying operating systems. The practice of scaling bioinformatics tools is increasingly commonplace. As such, this hands‐on application of refactoring techniques can serve as a valuable guide. Significantly, our customized Makeflow framework enabled generalizable deployment on a wider variety of systems while substantially reducing wall clock runtimes using hundreds of cores. Copyright © 2012 John Wiley &amp; Sons, Ltd.</p> </abstract>