Design Parameter Values for Impact Evaluations of Science and Mathematics Interventions Involving Teacher Outcomes. Issue 4 (1st October 2020)

• Record Type: Journal Article
• Authors: Westine, Carl D.; Unlu, Fatih; Taylor, Joseph; Spybrook, Jessaca; Zhang, Qi; Anderson, Brent
• Is Part Of: Journal of research on educational effectiveness. Volume 13:Issue 4(2020)
• Page Start: 816
• Page End: 839
• Publication Date: 2020-10-01
• Subjects: Experimental design -- design parameters -- teacher professional development -- science education -- intraclass correlation; Education -- Research -- Periodicals; Education -- Evaluation -- Periodicals; Education -- Methodology -- Periodicals; 371.422
• DOI: 10.1080/19345747.2020.1821849
• Languages: English
• ISSNs: 1934-5747

Abstract

Abstract: Experimental research in education and training programs typically involves administering treatment to whole groups of individuals. As such, researchers rely on the estimation of design parameter values to conduct power analyses to efficiently plan their studies to detect desired effects. In this study, we present design parameter estimates from a compilation of 11 studies involving teacher-level outcomes. These results expand upon what little is known about the clustering of variance among teacher-level outcomes. The findings inform the design of intervention studies which nest teachers within schools and aim to improve teacher mathematics and science content knowledge or instructional practices. The results show large differences in unconditional intraclass correlation coefficients across studies as well as within outcomes. They also quantify the relative importance of having a pretest measure to promote efficiency. This study highlights a need for improved reporting of this information in the literature to facilitate better experimental designs of interventions involving teacher-level outcomes.