A Bayesian Power Analysis Procedure Considering Uncertainty in Effect Size Estimates from a Meta-analysis. (2nd September 2016)

• Record Type: Journal Article
• Authors: Du, Han; Wang, Lijuan
• Is Part Of: Multivariate behavioral research. Volume 51:Number 5(2016)
• Page Start: 589
• Page End: 605
• Publication Date: 2016-09-02
• Subjects: Bayesian methods -- power analysis -- sample size determination -- uncertainty -- effect size estimates -- meta-analysis; Psychometrics -- Periodicals; Psychology, Experimental -- Periodicals; Psychology, Experimental; Psychometrics; Periodicals; 150.15195
• DOI: 10.1080/00273171.2016.1191324
• Languages: English
• ISSNs: 0027-3171

Abstract

ABSTRACT: In conventional frequentist power analysis, one often uses an effect size estimate, treats it as if it were the true value, and ignores uncertainty in the effect size estimate for the analysis. The resulting sample sizes can vary dramatically depending on the chosen effect size value. To resolve the problem, we propose a hybrid Bayesian power analysis procedure that models uncertainty in the effect size estimates from a meta-analysis. We use observed effect sizes and prior distributions to obtain the posterior distribution of the effect size and model parameters. Then, we simulate effect sizes from the obtained posterior distribution. For each simulated effect size, we obtain a power value. With an estimated power distribution for a given sample size, we can estimate the probability of reaching a power level or higher and the expected power. With a range of planned sample sizes, we can generate a power assurance curve. Both the conventional frequentist and our Bayesian procedures were applied to conduct prospective power analyses for two meta-analysis examples (testing standardized mean differences in example 1 and Pearson's correlations in example 2). The advantages of our proposed procedure are demonstrated and discussed.