Telling the story of complex change: an Impact Framework for the real world. (8th June 2021)

• Record Type: Journal Article
• Authors: Willett, Jo; Barclay, Michelle; Mukoro, Felix; Sweeney, Grace
• Is Part Of: International journal for quality in health care. Volume 33:Number 3(2021)
• Publication Date: 2021-06-08
• Subjects: evaluation -- quality improvement -- heath and care organizations -- research design -- research methods; Medical care -- Quality control -- Periodicals; 362.1068
• DOI: 10.1093/intqhc/mzab090
• Languages: English
• ISSNs: 1353-4505

Abstract

Abstract: Background: In the National Health Service (NHS) in England, traditional approaches to evidencing impact and value have an important role to play but are unlikely to demonstrate the full value of national quality improvement programmes and large-scale change initiatives in health and care. This type of work almost always takes place in complex and complicated settings, in that it involves multiple players, numerous interventions and a host of other confounding factors. Improvement work is usually emergent, with cause and effect only understood in hindsight; challenges around contribution and attribution can lead the key players to question how they can be certain that the described or observed changes are due to their intervention and would not have happened without them. In this complex environment, there is a risk of oversimplifying the observed impact and focusing instead on those things that are easier to measure, missing that which is important but more difficult to evidence. Methods: Between 2016 and 2019, an action-orientated approach, drawing on realist and development evaluation approaches, was taken to designing and testing the Impact Framework. First, we undertook a pragmatic review of tools and approaches used by others to capture and demonstrate their impact both within and outside the health and care environment. Following the identification and review of these tools and approaches, and in consultation with national improvement teams in England about their evaluation challenges and aspirations, we developed a set of underpinning principles to inform the design and build of the framework. The principles were informed and finessed following conversations with improvement teams and programme leads in NHS England with respect to the challenges that they were facing and their aspirations in terms of demonstrating their impact and learning as they worked. Results: The 'Impact Framework' described in this article offers a practical approach to capturing the impact of improvement work at any scale, taking account of unintended outcomes, considering attribution and contribution, and using a narrative approach to uncover the difference made by improvement initiatives in rich detail. In this article, we describe how the Impact Framework has been used with one of NHS England's national programmes, Time for Care, which was delivered between 2016 and 2020. Conclusions: The Impact Framework continues to be used, developed and further tested by national improvement programmes being delivered by NHS England and NHS Improvement and is updated regularly. The framework has been developed to be accessible to frontline teams and is supported by a set of resources to help improvement teams and individuals to use by themselves (https://www.england.nhs.uk/sustainableimprovement/impact-framework/ ).