Causal Inference with Multilevel Data: A Comparison of Different Propensity Score Weighting Approaches. (2nd November 2022)

• Record Type: Journal Article
• Authors: Fuentes, Alvaro; Lüdtke, Oliver; Robitzsch, Alexander
• Is Part Of: Multivariate behavioral research. Volume 57:Number 6(2022)
• Page Start: 916
• Page End: 939
• Publication Date: 2022-11-02
• Subjects: Causal inference -- propensity scores -- multilevel data -- weighting -- calibration weights; Psychometrics -- Periodicals; Psychology, Experimental -- Periodicals; Psychology, Experimental; Psychometrics; Periodicals; 150.15195
• DOI: 10.1080/00273171.2021.1925521
• Languages: English
• ISSNs: 0027-3171

Abstract

Abstract: Propensity score methods are a widely recommended approach to adjust for confounding and to recover treatment effects with non-experimental, single-level data. This article reviews propensity score weighting estimators for multilevel data in which individuals (level 1) are nested in clusters (level 2) and nonrandomly assigned to either a treatment or control condition at level 1. We address the choice of a weighting strategy (inverse probability weights, trimming, overlap weights, calibration weights) and discuss key issues related to the specification of the propensity score model (fixed-effects model, multilevel random-effects model) in the context of multilevel data. In three simulation studies, we show that estimates based on calibration weights, which prioritize balancing the sample distribution of level-1 and (unmeasured) level-2 covariates, should be preferred under many scenarios (i.e., treatment effect heterogeneity, presence of strong level-2 confounding) and can accommodate covariate-by-cluster interactions. However, when level-1 covariate effects vary strongly across clusters (i.e., under random slopes), and this variation is present in both the treatment and outcome data-generating mechanisms, large cluster sizes are needed to obtain accurate estimates of the treatment effect. We also discuss the implementation of survey weights and present a real-data example that illustrates the different methods.