Cased-based modelling and scenario simulation for ex-post evaluation. (January 2021)

• Record Type: Journal Article
• Authors: Schimpf, Corey; Barbrook-Johnson, Pete; Castellani, Brian
• Other Names: Barbrook-Johnson Pete guest-editor.; Castellani Brian guest-editor.; Hills Dione guest-editor.; Penn Alexandra guest-editor.; Gilbert Nigel guest-editor.
• Is Part Of: Evaluation. Volume 27:Number 1(2021)
• Page Start: 116
• Page End: 137
• Publication Date: 2021-01
• Subjects: case-based methods -- computational social science -- evaluation -- policy -- scenario analysis -- social complexity; Evaluation research (Social action programs) -- Periodicals; 361.61068
• DOI: 10.1177/1356389020978490
• Languages: English
• ISSNs: 1356-3890

Abstract

Despite 20 years of increasing acceptance, implementing complexity-appropriate methods for ex-post evaluation remains a challenge: instead of focusing on complex interventions, methods need to help evaluators better explore how policies (no matter how simple) take place in real-world, open, dynamic systems where many intertwined factors about the cases being targeted affect outcomes in numerous ways. To assist in this advance, we developed case-based scenario simulation, a new visually intuitive evaluation tool grounded in a data-driven, case-based, computational modelling approach, which evaluators can use to explore counterfactuals, status-quo trends, and what-if scenarios for some potential set of real or imagined interventions. To demonstrate the value and versatility of case-based scenario simulation we explore four published evaluations that differ in design (cross sectional, longitudinal, and experimental) and purpose (learning or accountability), and present a prospective view of how case-based scenario simulation could support and enhance evaluators' efforts in these complex contexts.