The role of machine learning in scientific workflows. (November 2019)

• Record Type: Journal Article
• Authors: Deelman, Ewa; Mandal, Anirban; Jiang, Ming; Sakellariou, Rizos
• Other Names: Dongarra Jack guest-editor.; Tourancheau Bernard guest-editor.
• Is Part Of: International journal of high performance computing applications. Volume 33:Number 6(2019)
• Page Start: 1128
• Page End: 1139
• Publication Date: 2019-11
• Subjects: Scientific workflows -- machine learning -- workflow systems -- anomaly detection -- workflow composition; High performance computing -- Periodicals; Supercomputers -- Periodicals; 004.1105
• DOI: 10.1177/1094342019852127
• Languages: English
• ISSNs: 1094-3420

Abstract

Machine learning (ML) is being applied in a number of everyday contexts from image recognition, to natural language processing, to autonomous vehicles, to product recommendation. In the science realm, ML is being used for medical diagnosis, new materials development, smart agriculture, DNA classification, and many others. In this article, we describe the opportunities of using ML in the area of scientific workflow management. Scientific workflows are key to today's computational science, enabling the definition and execution of complex applications in heterogeneous and often distributed environments. We describe the challenges of composing and executing scientific workflows and identify opportunities for applying ML techniques to meet these challenges by enhancing the current workflow management system capabilities. We foresee that as the ML field progresses, the automation provided by workflow management systems will greatly increase and result in significant improvements in scientific productivity.