File-based localization of numerical perturbations in data analysis pipelines. Issue 12 (2nd December 2020)

• Record Type: Journal Article
• Authors: Salari, Ali; Kiar, Gregory; Lewis, Lindsay; Evans, Alan C; Glatard, Tristan
• Is Part Of: GigaScience. Volume 9:Issue 12(2020)
• Publication Date: 2020-12-02
• Subjects: Reproducibility -- Operating Systems -- Neuroimaging -- Pipelines; Information storage and retrieval systems -- Research -- Periodicals; Biology -- Research -- Periodicals; Medical sciences -- Research -- Periodicals; Database management -- Periodicals; 570.285
• DOI: 10.1093/gigascience/giaa106
• Languages: English
• ISSNs: 2047-217X

Abstract

Abstract: Background: Data analysis pipelines are known to be affected by computational conditions, presumably owing to the creation and propagation of numerical errors. While this process could play a major role in the current reproducibility crisis, the precise causes of such instabilities and the path along which they propagate in pipelines are unclear. Method: We present Spot, a tool to identify which processes in a pipeline create numerical differences when executed in different computational conditions. Spot leverages system-call interception through ReproZip to reconstruct and compare provenance graphs without pipeline instrumentation. Results: By applying Spot to the structural pre-processing pipelines of the Human Connectome Project, we found that linear and non-linear registration are the cause of most numerical instabilities in these pipelines, which confirms previous findings.