Significance Tests and Estimates for R2 for Multiple Regression in Multiply Imputed Datasets: A Cautionary Note on Earlier Findings, and Alternative Solutions. (4th July 2019)

• Record Type: Journal Article
• Authors: van Ginkel, Joost R.
• Is Part Of: Multivariate behavioral research. Volume 54:Number 4(2019)
• Page Start: 514
• Page End: 529
• Publication Date: 2019-07-04
• Subjects: Missing data -- multiple imputation -- multiple regression -- coefficient of determination; Psychometrics -- Periodicals; Psychology, Experimental -- Periodicals; Psychology, Experimental; Psychometrics; Periodicals; 150.15195
• DOI: 10.1080/00273171.2018.1540967
• Languages: English
• ISSNs: 0027-3171

Abstract

Abstract: Whenever multiple regression is applied to a multiply imputed data set, several methods for combining significance tests for R 2 and the change in R 2 across imputed data sets may be used: the combination rules by Rubin, the Fisher z -test for R 2 by Harel, and F -tests for the change in R 2 by Chaurasia and Harel. For pooling R 2 itself, Harel proposed a method based on a Fisher z transformation. In the current article, it is argued that the pooled R 2 based on the Fisher z transformation, the Fisher z -test for R 2, and the F -test for the change in R 2 have some theoretical flaws. An argument is made for using Rubin's method for pooling significance tests for R 2 instead, and alternative procedures for pooling R 2 are proposed: simple averaging and a pooled R 2 constructed from the pooled significance test by Rubin. Simulations show that the Fisher z -test and Chaurasia and Harel's F -tests generally give inflated type-I error rates, whereas the type-I error rates of Rubin's method are correct. Of the methods for pooling the point estimates of R 2 no method clearly performs best, but it is argued that the average of R 2 's across imputed data set is preferred.