Matching Methods for Clustered Observational Studies in Education. Issue 3 (3rd July 2021)

• Record Type: Journal Article
• Authors: Keele, Luke; Lenard, Matthew; Page, Lindsay
• Is Part Of: Journal of research on educational effectiveness. Volume 14:Issue 3(2021)
• Page Start: 696
• Page End: 725
• Publication Date: 2021-07-03
• Subjects: Causal inference -- clustered observational studies -- clustered randomized trials -- hierarchical/multilevel data -- optimal matching; Education -- Research -- Periodicals; Education -- Evaluation -- Periodicals; Education -- Methodology -- Periodicals; 371.422
• DOI: 10.1080/19345747.2021.1875527
• Languages: English
• ISSNs: 1934-5747

Abstract

Abstract: Many interventions in education occur in settings where treatments are applied to groups. For example, a reading intervention may be implemented for all students in some schools and withheld from students in other schools. When such treatments are nonrandomly allocated, outcomes across the treated and control groups may differ due to the treatment or due to baseline differences between groups. When this is the case, researchers can use statistical adjustment to make treated and control groups similar in terms of observed characteristics. Recent work in statistics has developed matching methods designed for contexts where treatments are clustered. This form of matching, known as multilevel matching, may be well suited to many education applications where treatments are assigned to schools. In this article, we provide an extensive evaluation of multilevel matching and compare it to multilevel regression modeling. We evaluate multilevel matching methods in two ways. First, we use these matching methods in a within-study comparison design and attempt to recover treatment effect estimates from three clustered randomized trials . Second, we conduct a simulation study. We find evidence that generally favors an analytic approach to statistical adjustment that combines multilevel matching with regression adjustment. We conclude with an empirical application.