Rounding the Regression. Issue 3 (16th March 2018)

• Record Type: Journal Article
• Authors: Marland, Eric; Bossé, Michael J.; Rhoads, Gregory
• Is Part Of: PRIMUS. Volume 28:Issue 3(2018)
• Page Start: 187
• Page End: 205
• Publication Date: 2018-03-16
• Subjects: Rounding error -- regression -- significant figures -- statistical simulation -- least squares -- teaching; Mathematics -- Study and teaching (Higher) -- Periodicals; 510.711
• DOI: 10.1080/10511970.2017.1289289
• Languages: English
• ISSNs: 1051-1970

Abstract

Abstract: Rounding is a necessary step in many mathematical processes. We are taught early in our education about significant figures and how to properly round a number. So when we are given a data set and asked to find a regression line, we are inclined to offer the line with rounded coefficients to reflect our model. However, the effects are not as insignificant as they might seem at first. In this paper, we investigate some consequences of rounding the coefficients in a least squares linear regression with respect to the calculated value of R 2, and consider ways to minimize the amount of error that can arise.