Distributed X‐ray photon correlation spectroscopy data reduction using Hadoop MapReduce. (15th June 2018)

• Record Type: Journal Article
• Authors: Khan, Faisal; Narayanan, Suresh; Sersted, Roger; Schwarz, Nicholas; Sandy, Alec
• Is Part Of: Journal of synchrotron radiation. Volume 25:Part 4(2018)
• Page Start: 1135
• Page End: 1143
• Publication Date: 2018-06-15
• Subjects: multi‐speckle X‐ray photon correlation spectroscopy -- XPCS -- MapReduce -- parallel implementation -- Advanced Photon Source; Synchrotron radiation -- Periodicals; Free electron lasers -- Periodicals; 539.73505
• DOI: 10.1107/S160057751800601X
• Languages: English
• ISSNs: 0909-0495

Abstract

Abstract : MapReduce ‐based implementation of computing auto‐correlations for X‐ray photon correlation spectroscopy data at beamline 8‐ID at the Advanced Photon Source are presented. Abstract : Multi‐speckle X‐ray photon correlation spectroscopy (XPCS) is a powerful technique for characterizing the dynamic nature of complex materials over a range of time scales. XPCS has been successfully applied to study a wide range of systems. Recent developments in higher‐frame‐rate detectors, while aiding in the study of faster dynamical processes, creates large amounts of data that require parallel computational techniques to process in near real‐time. Here, an implementation of the multi‐tau and two‐time autocorrelation algorithms using the Hadoop MapReduce framework for distributed computing is presented. The system scales well with regard to the increase in the data size, and has been serving the users of beamline 8‐ID‐I at the Advanced Photon Source for near real‐time autocorrelations for the past five years.