Knowing the signs: a direct and generalizable motivation of two‐sided tests. (26th December 2019)

• Record Type: Journal Article
• Authors: Rice, Kenneth; Bonnett, Tyler; Krakauer, Chloe
• Is Part Of: Journal of the Royal Statistical Society. Volume 183:Number 2(2020)
• Page Start: 411
• Page End: 430
• Publication Date: 2019-12-26
• Subjects: Bayesian -- Decision theory, Frequentist -- Hypothesis tests -- Significance tests; Social sciences -- Statistical methods -- Periodicals; Statistics -- Periodicals; 300.15195
• DOI: 10.1111/rssa.12496
• Languages: English
• ISSNs: 0964-1998

Abstract

Summary: Many well‐known problems with two‐sided p ‐values are due to their use in hypothesis tests, with 'reject–accept' conclusions about point null hypotheses. We present an alternative motivation for p ‐value‐based tests, viewing them as assessments of only the sign of an underlying parameter, where we can conclude that the parameter is positive or negative, or simply say nothing either way. Our approach is decision theoretic, but—unusually—we consider the whole set of possible utility functions available. Doing this we show how, in a specific sense, close analogues of familiar one‐ and two‐sided tests are always the optimal decision. We argue that this simplicity could aid non‐experts' understanding and use of tests—and help them to think critically about whether or not tests are appropriate tools for answering their questions of interest. Several extensions are also considered, showing that the simple idea of determining the signs of parameters yields a rich framework for inference.