Analyzing the Influence of Dosage in Social Experiments With Application to the Supporting Healthy Marriage Program. (June 2018)

• Record Type: Journal Article
• Authors: Moulton, Shawn R.; Peck, Laura R.; Greeney, Adam
• Is Part Of: American journal of evaluation. Volume 39:Number 2(2018:Jun.)
• Page Start: 257
• Page End: 277
• Publication Date: 2018-06
• Subjects: program evaluation -- subgroup analysis -- social experiments -- dosage -- communication skills -- marriage; Evaluation -- Periodicals; Public administration -- Evaluation -- Periodicals; Educational evaluation -- Periodicals; Social sciences -- Methodology -- Periodicals; 300.72
• DOI: 10.1177/1098214017698566
• Languages: English
• ISSNs: 1098-2140

Abstract

In experimental evaluations of health and social programs, the role of dosage is rarely explored because researchers cannot usually randomize individuals to experience varying dosage levels. Instead, such evaluations reveal the average effects of exposure to an intervention, although program exposure may vary widely. This article compares three analytic methods that are used to examine dosage effects: the Analysis of Symmetrically Predicted Endogenous Subgroups (ASPES) method developed by Peck, instrumental variables, and propensity score matching methods. The article provides an illustration using data from the Supporting Healthy Marriage (SHM) demonstration, which rigorously tested the effectiveness of a skills-based relationship education program that had high take-up (91%) but substantial variation in exposure thereafter. To understand the relationship between program dosage level and impact magnitude, we use the ASPES method to create subgroups defined by the number of workshop hours couples attended and estimate impacts for those groups. We find that the SHM program impact on study members who received any program services is similar in magnitude to the impact on those study members who received a high or low-to-moderate dose of program services. For experimental evaluations of federal programs, the ASPES method shows promise as a way to investigate the relationship between treatment dosage and impact magnitude, a relationship crucial to designers and funders of public programs seeking to maximize program effectiveness.