Evaluation in the Real World: Decision Points and Rationales in Creating A Rigorous Study Designed to Convey Ecologically Valid Findings. Issue 3 (16th December 2020)

• Record Type: Journal Article
• Authors: Sullivan, Cris M.; Chiaramonte, Danielle; López‐Zerón, Gabriela; Gregory, Katie; Olsen, Linda
• Other Names: Wallerstein Nina guestEditor.; Christens Brian guestEditor.
• Is Part Of: American journal of community psychology. Volume 67:Issue 3/4(2021)
• Page Start: 447
• Page End: 455
• Publication Date: 2020-12-16
• Subjects: Program evaluation -- Domestic violence -- Intimate partner violence -- Randomized -- Ecological validity; Community psychology -- Periodicals; Community mental health services -- Periodicals; Community psychiatry -- Periodicals; Community Mental Health Services -- Periodicals; Community Psychiatry -- Periodicals; 155.905
• DOI: 10.1002/ajcp.12485
• Languages: English
• ISSNs: 0091-0562

Abstract

Abstract: Rigorously evaluating community‐based interventions for multiply marginalized populations is fraught with challenges under the best of circumstances. This manuscript describes the methodology chosen to evaluate an innovative model designed to help survivors of intimate partner violence obtain safe and stable housing. We justify the choice of evaluation design from a community psychology perspective and detail why we believe the multi‐method, multi‐source design, that also focuses on social context, will maximize ecological validity and, therefore, propel the scale‐up of the intervention if it is found to be effective. Longitudinal data are being collected from program recipients over time, the advocates who worked with them, agency service records, and monthly documentation of agency resources on hand that can impact services provided. Special attention is focused on capturing contextual information that can impact program success. While randomized control trials are still too often heralded as "the gold standard" for measuring intervention effectiveness, we maintain that the current design, which was developed in partnership with key community stakeholders, holds more promise when evaluating many community‐based programs. Highlights: Pragmatic evaluations, designed collaboratively with community partners, can increase ecological validity. Community psychologists can learn from concrete examples of pragmatic, real‐world evaluations. This manuscript describes the development of a multi‐method, longitudinal quasi‐experimental evaluation. The evaluation design includes a focus on how social context can impact program outcomes.