Comparing fidelity monitoring methods in an evidence-based parenting intervention. Issue 2 (11th February 2021)

• Record Type: Journal Article
• Authors: Tiwari, Ashwini; Whitaker, Daniel; Self-Brown, Shannon
• Is Part Of: Journal of children's services. Volume 16:Issue 2(2021)
• Page Start: 104
• Page End: 116
• Publication Date: 2021-02-11
• Subjects: Implementation -- Child maltreatment -- Fidelity -- Parent-training programs; Child care services -- Great Britain -- Periodicals; Social work with children -- Great Britain -- Periodicals; Child care services -- Periodicals; Child development -- Periodicals; Social work with children -- Periodicals; 362.705
• DOI: 10.1108/JCS-01-2020-0005
• Languages: English
• ISSNs: 1746-6660

Abstract

Abstract : Purpose: Two common methods in community settings of assessing program fidelity, a critical implementation component for program effectiveness, are video and audio recordings of sessions. This paper aims to examine how these two methods compared when used for a home-based behavioral parenting-training model (SafeCare®). Design/methodology/approach: Twenty-five SafeCare video-recorded sessions between home visitors and parents were scored by trained raters either using the video or audio-only portions of recordings. Sessions were coded using fidelity checklists, with items ( n = 33) classified as one of two fidelity aspects, content [delivery of program components ( n = 15)], or process [communication and rapport building ( n = 11)]. Seven items were considered to overlap between constructs. Items were coded as having been done or not done appropriately. Coders rated items as "technological limitation" when scoring methods hindered coding. Analyses compared percent agreement and disagreement between audio and video coders. Findings: Overall agreement between coders was 72.12%. Levels of agreement were higher for content items (M = 80.89%, SD = 19.68) than process items (58.54%, SD = 34.41). Disagreements due to technology limitations among audio coders were noted among 15 items; particularly, higher levels of disagreement were seen among process items (42.42%) than content items (9.64%). Originality/value: Compared to video, fidelity monitoring via audio recordings was associated with some loss of process-related fidelity. However, audio recordings could be sufficient with supplements such as participant surveys, to better capture process items. Research should also examine how content and process fidelity relate to changes in family behavior to further inform optimal fidelity monitoring methods for program use.