Making complex prediction rules applicable for readers: Current practice in random forest literature and recommendations. Issue 5 (1st August 2018)

• Record Type: Journal Article
• Authors: Boulesteix, Anne‐Laure; Janitza, Silke; Hornung, Roman; Probst, Philipp; Busen, Hannah; Hapfelmeier, Alexander
• Is Part Of: Biometrical journal. Volume 61:Issue 5(2019:Sep.)
• Page Start: 1314
• Page End: 1328
• Publication Date: 2018-08-01
• Subjects: logistic regression -- machine learning -- prediction rule -- reproducibility -- reproducible research; Biometry -- Periodicals; Medical statistics -- Periodicals; 570.15195
• DOI: 10.1002/bimj.201700243
• Languages: English
• ISSNs: 0323-3847

Abstract

Abstract: Ideally, prediction rules should be published in such a way that readers may apply them, for example, to make predictions for their own data. While this is straightforward for simple prediction rules, such as those based on the logistic regression model, this is much more difficult for complex prediction rules derived by machine learning tools. We conducted a survey of articles reporting prediction rules that were constructed using the random forest algorithm and published in PLOS ONE in 2014–2015 in the field "medical and health sciences", with the aim of identifying issues related to their applicability. Making a prediction rule reproducible is a possible way to ensure that it is applicable; thus reproducibility is also examined in our survey. The presented prediction rules were applicable in only 2 of 30 identified papers, while for further eight prediction rules it was possible to obtain the necessary information by contacting the authors. Various problems, such as nonresponse of the authors, hampered the applicability of prediction rules in the other cases. Based on our experiences from this illustrative survey, we formulate a set of recommendations for authors who aim to make complex prediction rules applicable for readers. All data including the description of the considered studies and analysis codes are available as supplementary materials.