Conquering big data with high performance computing. ([2016])

• Record Type: Book
• Further Information: Note: Ritu Arora, editor.
• Editors: Arora, Ritu
• Contents: Preface; Contents; 1 An Introduction to Big Data, High Performance Computing, High-Throughput Computing, and Hadoop; 1.1 Big Data; 1.2 High Performance Computing (HPC); 1.2.1 HPC Platform; 1.2.2 Serial and Parallel Processing on HPC Platform; 1.3 High-Throughput Computing (HTC); 1.4 Hadoop; 1.4.1 Hadoop-Related Technologies; 1.4.2 Some Limitations of Hadoop and Hadoop-Related Technologies; 1.5 Convergence of Big Data, HPC, HTC, and Hadoop; 1.6 HPC and Big Data Processing in Cloud and at Open-Science Data Centers; 1.7 Conclusion; References. 2 Using High Performance Computing for Conquering Big Data2.1 Introduction; 2.2 The Big Data Life Cycle; 2.3 Technologies and Hardware Platforms for Managing the Big Data Life Cycle; 2.4 Managing Big Data Life Cycle on HPC Platforms at Open-Science Data Centers; 2.4.1 TACC Resources and Usage Policies; 2.4.2 End-to-End Big Data Life Cycle on TACC Resources; 2.5 Use Case: Optimization of Nuclear Fusion Devices; 2.5.1 Optimization; 2.5.2 Computation on HPC; 2.5.3 Visualization Using GPUs; 2.5.4 Permanent Storage of Valuable Data; 2.6 Conclusions; References. 3 Data Movement in Data-Intensive High Performance Computing3.1 Introduction; 3.2 Node-Level Data Movement; 3.2.1 Case Study: ADAMANT; 3.2.2 Case Study: Energy Cost of Data Movement; 3.3 System-Level Data Movement; 3.3.1 Case Study: Graphs; 3.3.2 Case Study: Map Reduce; 3.4 Center-Level Data Movement; 3.4.1 Case Study: Spider; 3.4.2 Case Study: Gordon and Oasis; 3.5 About the Authors; References; 4 Using Managed High Performance Computing Systems for High-Throughput Computing; 4.1 Introduction; 4.2 What Are We Trying to Do?; 4.2.1 Deductive Computation; 4.2.2 Inductive Computation. 4.2.2.1 High-Throughput Computing4.3 Hurdles to Using HPC Systems for HTC; 4.3.1 Runtime Limits; 4.3.2 Jobs-in-Queue Limits; 4.3.3 Dynamic Job Submission Restrictions; 4.3.4 Solutions from Resource Managers and Big Data Research; 4.3.5 A Better Solution for Managed HPC Systems; 4.4 Launcher; 4.4.1 How Launcher Works; 4.4.2 Guided Example: A Simple Launcher Bundle; 4.4.2.1 Step 1: Create a Job File; 4.4.2.2 Step 2: Build a SLURM Batch Script; 4.4.3 Using Various Scheduling Methods; 4.4.3.1 Dynamic Scheduling; 4.4.3.2 Static Scheduling; 4.4.4 Launcher with Intel®Xeon Phi Coprocessors. 4.4.4.1 Offload4.4.4.2 Independent Workloads for Host and Coprocessor; 4.4.4.3 Symmetric Execution on Host and Phi; 4.4.5 Use Case: Molecular Docking and Virtual Screening; 4.5 Conclusion; References; 5 Accelerating Big Data Processing on Modern HPC Clusters; 5.1 Introduction; 5.2 Overview of Apache Hadoop and Spark; 5.2.1 Overview of Apache Hadoop Distributed File System; 5.2.2 Overview of Apache Hadoop MapReduce; 5.2.3 Overview of Apache Spark; 5.3 Overview of High-Performance Interconnects and Storage Architecture on Modern HPC Clusters.
• Publisher Details: Switzerland : Springer
• Publication Date: 2016
• Extent: 1 online resource
• Subjects: 004.11; Computer science; High performance computing; Big data; COMPUTERS -- Computer Literacy; COMPUTERS -- Computer Science; COMPUTERS -- Data Processing; COMPUTERS -- Hardware -- General; COMPUTERS -- Information Technology; COMPUTERS -- Machine Theory; COMPUTERS -- Reference; Big data; High performance computing; Computers -- Data Modeling & Design; Systems analysis & design; Algorithms & data structures; Database management; Computer network architectures; Data structures (Computer science); Computers -- Database Management -- General; Databases; Electronic books
• Languages: English
• ISBNs: 9783319337425; 3319337424
• Related ISBNs: 9783319337401; 3319337408
• Notes: Note: Includes bibliographical references.; Note: Online resource; title from PDF title page (EBSCO, viewed October 26, 2016).
• Access Rights: Legal Deposit; Only available on premises controlled by the deposit library and to one user at any one time; The Legal Deposit Libraries (Non-Print Works) Regulations (UK).
• Access Usage: Restricted: Printing from this resource is governed by The Legal Deposit Libraries (Non-Print Works) Regulations (UK) and UK copyright law currently in force.