The role of treatment acceptability, effectiveness, and understanding in treatment fidelity: Predicting implementation variation in a middle school science program. (December 2015)

• Record Type: Journal Article
• Authors: Lakin, Joni M.; Shannon, David M.
• Is Part Of: Studies in educational evaluation. Volume 47(2015)
• Page Start: 28
• Page End: 37
• Publication Date: 2015-12
• Subjects: PCK pedagogical content knowledge -- POSITT Pedagogy of Science Inquiry Teaching Test -- STEM science, technology, engineering, and mathematics -- SECME South Eastern Consortium for Minorities in Engineering; Program evaluation -- School-based evaluation -- Evaluation methods -- Fidelity of implementation; Curriculum evaluation -- Periodicals; Educational tests and measurements -- Periodicals; Programmes d'études -- Evaluation -- Périodiques; Tests et mesures en éducation -- Périodiques; 375.006
• DOI: 10.1016/j.stueduc.2015.06.002
• Languages: English
• ISSNs: 0191-491X

Abstract

Highlights: Teacher variation in implementing school-based programs complicates evaluation. Implementation may be affected by acceptability, effectiveness, and understanding. Newly developed assessments gauge these constructs for school-based intervention. Assessments showed modest associations with implementation of the program. Measures show promise for evaluation and research on implementation fidelity. Abstract: Evaluating a school-based program is a particular challenge when teachers implementing the program. Variations in implementation can be difficult to measure and predict. We developed measures to explore variation in treatment implementation and serve as predictors of variation in a school-based science education program. Based on previous work, we focused on assessing treatment acceptability, effectiveness, and understanding among teachers as critical determinants of variations in program implementation using self-reported and objective measures of implementation. All three constructs were associated with implementation of the program. Our measures of these constructs show promise for use in formative and summative evaluations. Our stratification of program elements with implementation predictors can provide a template for future exploration of treatment fidelity.