Misuse of the sign test in narrative synthesis of evidence. (15th July 2020)

• Record Type: Journal Article
• Authors: Nikolakopoulos, Stavros
• Is Part Of: Research synthesis methods. Volume 11:Number 5(2020)
• Page Start: 714
• Page End: 719
• Publication Date: 2020-07-15
• Subjects: sign test -- narrative synthesis -- binomial distribution -- Poisson‐binomial distribution; Research -- Methodology -- Periodicals; Research -- Statistical methods -- Periodicals; 507.2
• DOI: 10.1002/jrsm.1427
• Languages: English
• ISSNs: 1759-2879

Abstract

Abstract : In narrative synthesis of evidence, it can be the case that the only quantitative measures available concerning the efficacy of an intervention is the direction of the effect, that is, whether it is positive or negative. In such situations, the sign test has been proposed in the literature and in recent Cochrane guidelines as a way to test whether the proportion of positive effects is favorable. I argue that the sign test is inappropriate in this context as the data are not generated according to the binomial distribution it employs. I demonstrate possible consequences for both hypothesis testing and estimation via hypothetical examples.