Partially Identified Treatment Effects for Generalizability. Issue 3 (3rd July 2017)

• Record Type: Journal Article
• Authors: Chan, Wendy
• Is Part Of: Journal of research on educational effectiveness. Volume 10:Issue 3(2017)
• Page Start: 646
• Page End: 669
• Publication Date: 2017-07-03
• Subjects: causal inference -- generalizability -- partial identification -- bounds; Education -- Research -- Periodicals; Education -- Evaluation -- Periodicals; Education -- Methodology -- Periodicals; 371.422
• DOI: 10.1080/19345747.2016.1273412
• Languages: English
• ISSNs: 1934-5747

Abstract

ABSTRACT: Recent methods to improve generalizations from nonrandom samples typically invoke assumptions such as the strong ignorability of sample selection, which is challenging to meet in practice. Although researchers acknowledge the difficulty in meeting this assumption, point estimates are still provided and used without considering alternative assumptions. We compare the point identifying assumption of strong ignorability of sample selection with two alternative assumptions—bounded sample variation and monotone treatment response—that partially identify the parameter of interest, yielding interval estimates. Additionally, we explore the role that population data frames play in contributing identifying power for the interval estimates. We situate the comparison around causal generalization with nonrandom samples by applying the assumptions to a cluster randomized trial in education. Bounds on the population average treatment effect are derived under the alternative assumptions and the case when no assumptions are made on the data. While comparing the bounds, we discuss the plausibility of each alternative assumption and the practical trade-offs. We highlight the importance of thoughtfully considering the role that assumptions play in causal generalization by illustrating the differences in inferences from different assumptions.