Informing Estimates of Program Effects for Studies of Mathematics Professional Development Using Teacher Content Knowledge Outcomes. (October 2016)

• Record Type: Journal Article
• Authors: Phelps, Geoffrey; Kelcey, Benjamin; Jones, Nathan; Liu, Shuangshuang
• Other Names: Kelcey Ben guest-editor.; Spybrook Jessaca guest-editor.
• Is Part Of: Evaluation review. Volume 40:Number 5(2016)
• Page Start: 383
• Page End: 409
• Publication Date: 2016-10
• Subjects: mathematics -- professional development -- mathematical knowledge for teaching -- program evaluation -- group randomized trial -- program effects; Evaluation research (Social action programs) -- Periodicals; 361.0072
• DOI: 10.1177/0193841X16665024
• Languages: English
• ISSNs: 0193-841X

Abstract

Mathematics professional development is widely offered, typically with the goal of improving teachers' content knowledge, the quality of teaching, and ultimately students' achievement. Recently, new assessments focused on mathematical knowledge for teaching (MKT) have been developed to assist in the evaluation and improvement of mathematics professional development. This study presents empirical estimates of average program change in MKT and its variation with the goal of supporting the design of experimental trials that are adequately powered to detect a specified program effect. The study drew on a large database representing five different assessments of MKT and collectively 326 professional development programs and 9, 365 teachers. Results from cross-classified hierarchical growth models found that standardized average change estimates across the five assessments ranged from a low of 0.16 standard deviations ( SD s) to a high of 0.26 SD s. Power analyses using the estimated pre- and posttest change estimates indicated that hundreds of teachers are needed to detect changes in knowledge at the lower end of the distribution. Even studies powered to detect effects at the higher end of the distribution will require substantial resources to conduct rigorous experimental trials. Empirical benchmarks that describe average program change and its variation provide a useful preliminary resource for interpreting the relative magnitude of effect sizes associated with professional development programs and for designing adequately powered trials.