Evaluation Methodologies for Estimating the Likelihood of Program Implementation Failure. (September 2014)

• Record Type: Journal Article
• Authors: Durand, Roger; Decker, Phillip J.; Kirkman, Dorothy M.
• Is Part Of: American journal of evaluation. Volume 35:Number 3(2014:Sep.)
• Page Start: 404
• Page End: 418
• Publication Date: 2014-09
• Subjects: evaluation methodology -- failures in implementation -- implementation -- predicting failures; Evaluation -- Periodicals; Public administration -- Evaluation -- Periodicals; Educational evaluation -- Periodicals; Social sciences -- Methodology -- Periodicals; 300.72
• DOI: 10.1177/1098214014523824
• Languages: English
• ISSNs: 1098-2140

Abstract

Despite our best efforts as evaluators, program implementation failures abound. A wide variety of valuable methodologies have been adopted to explain and evaluate the why of these failures. Yet, typically these methodologies have been employed concurrently (e.g., project monitoring) or to the post-hoc assessment of program activities. What we believe to be missing are methods that will lead to the successful prediction of program implementation failures in advance, methods that will lead us directly to the "how" and, especially, the " how likely " of program implementation failure . To that end we propose, discuss, and illustrate three such methods that seemingly hold promise – marker analysis, the wisdom of crowds, and "Big Data." Additionally, we call for an expanded role for evaluation – explanation, but also prediction without a total embrace of the need to understand why a prediction works.