A Comparison of Model-Based and Design-Based Impact Evaluations of Interventions in Developing Countries. (September 2013)

• Record Type: Journal Article
• Authors: Hansen, Henrik; Klejnstrup, Ninja Ritter; Andersen, Ole Winckler
• Is Part Of: American journal of evaluation. Volume 34:Number 3(2013)
• Page Start: 320
• Page End: 338
• Publication Date: 2013-09
• Subjects: development -- impact -- nonexperimental -- social experiment -- within-study; Evaluation -- Periodicals; Public administration -- Evaluation -- Periodicals; Educational evaluation -- Periodicals; Social sciences -- Methodology -- Periodicals; 300.72
• DOI: 10.1177/1098214013476915
• Languages: English
• ISSNs: 1098-2140

Abstract

There is a long-standing debate as to whether nonexperimental estimators of causal effects of social programs can overcome selection bias. Most existing reviews either are inconclusive or point to significant selection biases in nonexperimental studies. However, many of the reviews, the so-called between-studies, do not make direct comparisons of the estimates. We survey four impact studies, all using data from development interventions that directly compare experimental and nonexperimental impact estimates. Our review illustrates that when the program participation process is well understood, and correctly modeled, then the nonexperimental estimators can overcome the selection bias to the same degree as randomized controlled trials. Hence, we suggest that evaluators of development programs aim to be careful and precise in the formulation of the statistical model for the assignment into the program and also to use the assignment information for model-based systematic sampling.