Measuring intervention fidelity from different perspectives with multiple methods: The Reflect program as an example. (December 2015)

• Record Type: Journal Article
• Authors: Schultes, Marie-Therese; Jöstl, Gregor; Finsterwald, Monika; Schober, Barbara; Spiel, Christiane
• Is Part Of: Studies in educational evaluation. Volume 47(2015)
• Page Start: 102
• Page End: 112
• Publication Date: 2015-12
• Subjects: Implementation -- Evaluation -- Intervention fidelity -- Self-determination theory -- Gender competence; Curriculum evaluation -- Periodicals; Educational tests and measurements -- Periodicals; Programmes d'études -- Evaluation -- Périodiques; Tests et mesures en éducation -- Périodiques; 375.006
• DOI: 10.1016/j.stueduc.2015.10.001
• Languages: English
• ISSNs: 0191-491X

Abstract

Highlights: We evaluate the implementation of a school-based gender competence project. We assess intervention fidelity from two different perspectives. We measure intervention fidelity with two different methods. We link intervention fidelity to program outcomes using multilevel analysis. Intervention fidelity from different perspectives predicts different outcomes. Abstract: Implementing programs with high intervention fidelity generally increases program effectiveness. When evaluating intervention fidelity, collecting data from multiple informants and using different methods is likely to clarify different aspects of program outcomes. The present study investigated intervention fidelity in a gender competence program being implemented by 31 teachers in their classrooms. Intervention fidelity was assessed by expert ratings of the teachers' project portfolios and a survey of 564 participating students. Multilevel analyses showed that the two perspectives predicted different outcomes: the experts' ratings were linked to the students' gain in knowledge, while the students' ratings were linked to a change in their perception of a diversity-fair classroom environment. The results argue for a multi-perspective, multi-method approach when evaluating intervention fidelity.