Evaluation of a scientific data search infrastructure. (19th October 2022)

• Record Type: Journal Article
• Authors: Orhean, Alexandru Iulian; Giannakou, Anna; Antypas, Katie; Raicu, Ioan; Ramakrishnan, Lavanya
• Is Part Of: Concurrency and computation. Volume 34:Number 27(2022)
• Page Start: n/a
• Page End: n/a
• Publication Date: 2022-10-19
• Subjects: scientific data -- search engine; Parallel processing (Electronic computers) -- Periodicals; Parallel computers -- Periodicals; 004.35
• DOI: 10.1002/cpe.7261
• Languages: English
• ISSNs: 1532-0626

Abstract

Summary: The ability to search over large scientific datasets has become crucial to next‐generation scientific discoveries as data generated from scientific facilities grow dramatically. In previous work, we developed and deployed ScienceSearch, a search infrastructure for scientific data which uses machine learning to automate metadata creation. Our current deployment is deployed atop a container based platform at a HPC center. In this article, we present an evaluation and discuss our experiences with the ScienceSearch infrastructure. Specifically, we present a performance evaluation of ScienceSearch's infrastructure focusing on scalability trends. The obtained results show that ScienceSearch is able to serve up to 130 queries/min with latency under 3 s. We discuss our infrastructure setup and evaluation results to provide our experiences and a perspective on opportunities and challenges of our search infrastructure.