A framework to assess the value of application of formal criteria to check clinical relevance in RCTs as part of a benefit assessment strategy. Issue 1 (14th August 2015)

• Record Type: Journal Article
• Authors: Vach, Werner; Gladstone, Beryl Primrose
• Is Part Of: Biometrical journal. Volume 58:Issue 1(2016:Jan.)
• Page Start: 59
• Page End: 75
• Publication Date: 2015-08-14
• Subjects: Clinical relevance -- Power -- RCT -- Sample size -- Shifted null hypothesis; Biometry -- Periodicals; Medical statistics -- Periodicals; 570.15195
• DOI: 10.1002/bimj.201300246
• Languages: English
• ISSNs: 0323-3847

Abstract

Abstract : Recently, the topic of assessing clinical relevance on top of statistical significance in the analysis of randomized control trials (RCTs) has got increasing attention, in particular as part of benefit assessments. Several formal criteria to serve this purpose have been published. In this paper, we present a framework to assess the value of the application of such criteria. We propose to quantify the need for the assessment of clinical relevance by the actual risk of having accepted a benefit for a treatment with an irrelevant effect in a successful RCT. We then study how this risk can be controlled by two popular criteria based on comparing the effect estimate or the lower bound of the confidence interval with a given threshold. We further propose to quantify the impact of using formal criteria by considering the expected costs when specifying error‐specific costs for each of the three possible types of errors: A benefit may be accepted for a treatment, which is actually inferior, or which is not inferior, but only implies an irrelevant improvement, or a benefit may be rejected for a treatment implying a relevant improvement. This way we can demonstrate that the impact depends on parameters which are typically not explicitly defined in the frame of benefit assessments. Depending on the values of these parameters, formal checks of clinical relevance may imply better decisions on average, but they may also imply more harm than good on average.