A collaborative analysis framework for distributed gridded environmental data. (January 2019) - Xu, Hao; Li, Sha; Bai, Yuqi; Dong, Wenhao; Huang, Wenyu; Xu, Shiming; Lin, Yanluan; Wang, Bin; Wu, Fanghua; Xin, Xiaoge; Zhang, Li; Wang, Zaizhi; et al.
Abstract: As the amount of environmental data expands exponentially worldwide, researchers are challenged to efficiently analyze data maintained in multiple data centers. Because distributed data access, server-side analysis, multinode collaboration, and extensible analytic functions are still research gaps in this field, this paper introduces a collaborative analysis framework for gridded environmental data, i.e. CAFE. Multiple CAFE nodes can collaborate to perform complex data analysis. Analytic functions are performed near where data are stored. A web-based user interface allows researchers to search for data of interest, submit analytic tasks, check the status of tasks, visualize the analysis results, and download the resulting data files. CAFE facilitates overall research efficiency by dramatically lowering the amount of data that must be transmitted from data centers to researchers for analysis. The results of this study may lead to the further development of collaborative computing paradigm for environmental data analysis. Highlights: CAFE enables data discovery, visualization, and analysis through a web-based portal. CAFE always execute data analysis near where data are stored. CAFE enables multiple nodes to collaboratively fulfill analysis task automatically. CAFE can alleviate large data transfer from data centers to researchers. CAFE can wrap executable analytic scripts as analytic functions for users to select.