A survey of numerical linear algebra methods utilizing mixed-precision arithmetic. (July 2021)

• Record Type: Journal Article
• Authors: Abdelfattah, Ahmad; Anzt, Hartwig; Boman, Erik G; Carson, Erin; Cojean, Terry; Dongarra, Jack; Fox, Alyson; Gates, Mark; Higham, Nicholas J; Li, Xiaoye S; Loe, Jennifer; Luszczek, Piotr; Pranesh, Srikara; Rajamanickam, Siva; Ribizel, Tobias; Smith, Barry F; Swirydowicz, Kasia; Thomas, Stephen; Tomov, Stanimire; Tsai, Yaohung M; Yang, Ulrike Meier
• Is Part Of: International journal of high performance computing applications. Volume 35:Number 4(2021)
• Page Start: 344
• Page End: 369
• Publication Date: 2021-07
• Subjects: Mixed-precision arithmetic -- numerical mathematics -- linear algebra -- high-performance computing -- GPUs; High performance computing -- Periodicals; Supercomputers -- Periodicals; 004.1105
• DOI: 10.1177/10943420211003313
• Languages: English
• ISSNs: 1094-3420

Abstract

The efficient utilization of mixed-precision numerical linear algebra algorithms can offer attractive acceleration to scientific computing applications. Especially with the hardware integration of low-precision special-function units designed for machine learning applications, the traditional numerical algorithms community urgently needs to reconsider the floating point formats used in the distinct operations to efficiently leverage the available compute power. In this work, we provide a comprehensive survey of mixed-precision numerical linear algebra routines, including the underlying concepts, theoretical background, and experimental results for both dense and sparse linear algebra problems.