Quasi-experimental evaluation using confirmatory procedures: A case study of an economic and social empowerment intervention to reduce girls' vulnerability to HIV in rural Mozambique. (December 2019)

• Record Type: Journal Article
• Authors: Burke, Holly McClain; Field, Samuel; González-Calvo, Lázaro; Eichleay, Margaret A.; Moon, Troy D.
• Is Part Of: Evaluation and program planning. Volume 77(2019)
• Publication Date: 2019-12
• Subjects: GBV Gender-based violence -- HIV Human Immunodeficiency Virus -- SCIP Strengthening Communities through Integrated Programming -- WF Women First -- USAID United States Agency for International Development; HIV prevention -- Economic strengthening -- Multi-sector -- Integrated development -- Adolescents -- Girls -- Mozambique -- Gender-based violence -- School attendance -- Sexual behavior -- Quasi-experimental; Health planning -- Periodicals; Medical care -- Evaluation -- Periodicals; 362.1068
• DOI: 10.1016/j.evalprogplan.2019.101721
• Languages: English
• ISSNs: 0149-7189

Abstract

Highlights: Quasi-experimental studies can be conducted using rigorous confirmatory procedures. Confirmatory procedures require a priori study design and analytic decision-making. We found no evidence of impact of the intervention on a priori selected quantitative outcomes. We describe how real-world challenges affected attaining results. Abstract: Purpose: Our goal was to determine whether an economic and social empowerment intervention implemented in Zambézia Province, Mozambique reduced girls' vulnerability to HIV. We use this experience to discuss challenges of evaluating real-world interventions. Methods: Two rounds of data were collected from 885 girls, 13–19 years, for this clustered, non-equivalent (two-stage) cohort trial. We used multi-level exact matching and difference-in-differences estimation to estimate intervention effects on two outcomes: girls' knowledge of gender-based violence and school attendance. Results: Estimates of two outcomes analysed indicated no statistically significant intervention effects. Preliminary analysis of data from the intervention group revealed this study was unable to obtain accurate measures for five outcomes related to HIV vulnerability. Conclusions: Although our study did not find evidence of impact on the a priori selected outcomes, we report on our experience implementing this robust methodologic design and describe how the challenges encountered in this program setting affected our ability to attain results. We recommend prospective evaluation designs with random allocation be accommodated early during planning. When not possible, quasi-experimental studies should collect data from large samples. To reduce measurement bias, biological endpoints such sexually transmitted infections should serve as primary outcomes for programs intending to reduce sexual behaviors.