A dataflow runtime environment and static scheduler for edge, fog and in-situ computing. (15th May 2019)

• Record Type: Journal Article
• Authors: Carvalho, Caio B.G.; Ferreira, Victor C.; França, Felipe M.G.; Bentes, Cristiana B.; Mencagli, Gabriele; Alves, Tiago A.O.; Sena, Alexandre C.; Marzulo, Leandro A.J.
• Is Part Of: International journal of grid and utility computing. Volume 10:Number 3(2019)
• Page Start: 235
• Page End: 247
• Publication Date: 2019-05-15
• Subjects: dataflow computing -- edge computing -- fog computing -- scheduling techniques -- smart storage; Electronic data processing -- Distributed processing -- Periodicals; Electronic commerce -- Management -- Computer programs -- Periodicals; 004.605
• Languages: English
• ISSNs: 1741-847X

Abstract

In the dataflow computation model, tasks are executed according to data dependencies, instead of following program order, enabling natural parallelism exploitation. Sucuri is a dataflow library for Python that allows transparent execution of applications on clusters of multicores, while taking care of scheduling issues. Recent trends in edge/fog/In-situ computing assume that storage and network devices will have processing elements with lower power consumption and performance, which would make a good case for runtime environments that deal with the data versus computation movements trade-off in a more transparent and automated way. This work presents a study on different factors that should be considered when running dataflow applications in in-situ environments, using Sucuri to conduct experiments in a small system emulating a smart storage (in-situ device) utilisation. A static scheduling solution is also presented, allowing Sucuri to choose the most suited approach regarding this in-situ trade-off.