Program Value-Added: A Feasible Method for Providing Evidence on the Effectiveness of Multiple Programs Implemented Simultaneously in Schools. (December 2022)

• Record Type: Journal Article
• Authors: Shand, Robert; M. Leach, Stephen; M. Hollands, Fiona; Chang, Florence; Pan, Yilin; Yan, Bo; Dossett, Dena; Nayyer-Qureshi, Samreen; Wang, Yixin; Head, Laura
• Is Part Of: American journal of evaluation. Volume 43:Number 4(2022)
• Page Start: 584
• Page End: 606
• Publication Date: 2022-12
• Subjects: data analysis -- impact evaluation -- PreK–12 education -- quasi-experiment; Evaluation -- Periodicals; Public administration -- Evaluation -- Periodicals; Educational evaluation -- Periodicals; Social sciences -- Methodology -- Periodicals; 300.72
• DOI: 10.1177/10982140211071017
• Languages: English
• ISSNs: 1098-2140

Abstract

We assessed whether an adaptation of value-added analysis (VAA) can provide evidence on the relative effectiveness of interventions implemented in a large school district. We analyzed two datasets, one documenting interventions received by underperforming students, and one documenting interventions received by students in schools benefiting from discretionary funds to invest in specific programs. Results from the former dataset identified several interventions that appear to be more or less effective than the average intervention. Results from the second dataset were counterintuitive. We conclude that, under specific conditions, program VAA can provide evidence to help guide district decision-makers to identify outlier interventions and inform decisions about scaling up or disinvesting in such interventions, with the caveat that if those conditions are not met, the results could be misleading.