A meta-framework for conducting mixed methods impact evaluations: Implications for altering practice and the teaching of evaluation. (June 2017)

• Record Type: Journal Article
• Authors: Onwuegbuzie, Anthony J.; Hitchcock, John H.
• Is Part Of: Studies in educational evaluation. Volume 53(2017)
• Page Start: 55
• Page End: 68
• Publication Date: 2017-06
• Subjects: Program evaluation -- Mixed methods -- Mixed research -- Impact evaluations -- Mixed Methods Theory-Based Impact Evaluations -- Rationale and purpose model -- Integration -- Meta-evaluation -- Rigorous evaluation -- Process analysis -- Credible counterfactual -- Counterfactual analysis -- Participant-oriented evaluations -- Authenticity criteria -- Mixed methods legitimation -- Debriefing; Curriculum evaluation -- Periodicals; Educational tests and measurements -- Periodicals; Programmes d'études -- Evaluation -- Périodiques; Tests et mesures en éducation -- Périodiques; 375.006
• DOI: 10.1016/j.stueduc.2017.02.001
• Languages: English
• ISSNs: 0191-491X

Abstract

Highlights: An evaluation meta-framework should be comprehensive, flexible, and meet enhanced complexity. Impact evaluation is the rigorous and systematic analysis that is conducted collaboratively. A comprehensive impact evaluation involves the use of both quantitative and qualitative data. Mixed Methods Theory-Based Impact Evaluations involve using mixed methods at all eight phases. Abstract: Educational interventions are becoming increasingly more complex, far-reaching, and high-stakes. Thus, there is a need for an evaluation meta-framework that is comprehensive, flexible, and meets enhanced complexity. Therefore, we provide a new and comprehensive definition of impact evaluations—what we call a comprehensive impact evaluation—that draws out the importance of collecting and analyzing both quantitative and qualitative data, thereby resulting in a rigorous approach that can allow for strong inferences. Further, we provide an overview of impact evaluation designs that can be used in comprehensive impact evaluations along with a rationale for using both quantitative and qualitative research techniques. This leads to the central purpose of the article: to provide a meta-framework for conducting what we call a Mixed Methods Theory-Based Impact Evaluations, wherein mixed methods techniques are used at every phase of the process. Building on White's (2009) work, we outline an 8-phase Mixed Methods Theory-Based Impact Evaluation.