Reporting health care decision models: a prospective reliability study of a multidimensional evaluation framework. (2nd September 2016)

• Record Type: Journal Article
• Authors: Müller, Dirk; Gerber-Grote, Andreas; Stollenwerk, Björn; Stock, Stephanie; Auweiler, Philipp W. P.; Frey, Simon; Adarkwah, Charles Christian; de Kinderen, Reina; Hellmich, Martin
• Is Part Of: Expert review of pharmacoeconomics & outcomes research. Volume 16:Number 5(2016)
• Page Start: 619
• Page End: 627
• Publication Date: 2016-09-02
• Subjects: Inter-rater reliability -- methodological assessment -- modeling studies -- intra class-correlation -- cost–effectiveness; Pharmacology -- Economic aspects -- Periodicals; 338.47615105
• DOI: 10.1586/14737167.2016.1115721
• Languages: English
• ISSNs: 1473-7167

Abstract

ABSTRACT: The aim of this study was to evaluate the inter-rater reliability of the Phillips-checklist, a proposed framework for the quality assessment of modeling studies. Six raters evaluated nine modeling studies from three different medical specialties. Intra-class correlation (ICC) and corresponding variance components were estimated from these studies. Raters were asked to comment on their experience with the framework. While overall the mean inter-rater reliability showed no significant rater-effect (ICC = 0.69, p = 0.064), there was – presumably as a result of a lower study variability – a significant rater effect for clopidogrel only ( p < 0.001). The framework allowed a more structured methodological assessment but several items remained unclear. Regarding the quality assessment of modeling studies with the proposed framework, the rater variability is similar or even higher than variability because of studies or residual effects. Several scoring items can and should be improved to ease interpretation.