Stochastic rounding: implementation, error analysis and applications. Issue 3 (9th March 2022)

• Record Type: Journal Article
• Authors: Croci, Matteo; Fasi, Massimiliano; Higham, Nicholas J.; Mary, Theo; Mikaitis, Mantas
• Is Part Of: Royal Society open science. Volume 9:Issue 3(2022)
• Publication Date: 2022-03-09
• Subjects: floating-point arithmetic -- rounding error analysis -- IEEE 754 -- binary16 -- bfloat16 -- machine learning; Science -- Periodicals; 500
• DOI: 10.1098/rsos.211631
• Languages: English
• ISSNs: 2054-5703

Abstract

Abstract : Stochastic rounding (SR) randomly maps a real number x to one of the two nearest values in a finite precision number system. The probability of choosing either of these two numbers is 1 minus their relative distance to x . This rounding mode was first proposed for use in computer arithmetic in the 1950s and it is currently experiencing a resurgence of interest. If used to compute the inner product of two vectors of length n in floating-point arithmetic, it yields an error bound with constant n u with high probability, where u is the unit round-off. This is not necessarily the case for round to nearest (RN), for which the worst-case error bound has constant nu . A particular attraction of SR is that, unlike RN, it is immune to the phenomenon of stagnation, whereby a sequence of tiny updates to a relatively large quantity is lost. We survey SR by discussing its mathematical properties and probabilistic error analysis, its implementation, and its use in applications, with a focus on machine learning and the numerical solution of differential equations.