Applications of Small Area Estimation to Generalization With Subclassification by Propensity Scores. (April 2018)

• Record Type: Journal Article
• Authors: Chan, Wendy
• Is Part Of: Journal of educational and behavioral statistics. Volume 43:Number 2(2018)
• Page Start: 182
• Page End: 224
• Publication Date: 2018-04
• Subjects: educational policy -- evaluation -- experimental design -- experimental research -- generalizability theory -- hierarchical linear modeling -- statistics; Educational statistics -- Periodicals; Social sciences -- Statistical methods -- Periodicals; 370.2
• DOI: 10.3102/1076998617733828
• Languages: English
• ISSNs: 1076-9986

Abstract

Policymakers have grown increasingly interested in how experimental results may generalize to a larger population. However, recently developed propensity score–based methods are limited by small sample sizes, where the experimental study is generalized to a population that is at least 20 times larger. This is particularly problematic for methods such as subclassification by propensity score, where limited sample sizes lead to sparse strata. This article explores the potential of small area estimation methods to improve the precision of estimators in sparse strata using population data as a source of auxiliary information to borrow strength. Results from simulation studies identify the conditions under which small area estimators outperform conventional estimators and the limitations of this application to causal generalization studies.