Deconstructing impact: A framework for impact evaluation in grant applications. (24th November 2021)

• Record Type: Journal Article
• Authors: Ma, Lai; Agnew, Rachael
• Is Part Of: Science and public policy. Volume 49:Number 2(2022)
• Page Start: 289
• Page End: 301
• Publication Date: 2021-11-24
• Subjects: ex ante impact assessment -- impact evaluation -- peer review -- societal impact; Science and state -- Periodicals; 338.926
• DOI: 10.1093/scipol/scab080
• Languages: English
• ISSNs: 0302-3427

Abstract

Abstract: Impact assessments in grant applications can be biased, random, or inconsistent. One reason is that there is not a framework to assist the review process. To develop fair and transparent evaluative criteria, it is necessary to understand what kinds of outcome and impact can reasonably be achieved and expected. Using content analysis, 100 UK Research Excellence Framework (REF) 2014 impact case studies were analysed based on the definition of the output, outcome, and impact in the logic model. The analysis shows that (1) outcomes and medium-term impacts, not long-term impacts, are reported in the case studies, (2) impacts can be categorised as use and experience based, and (3) there is a need to recognise the creation and co-creation of impacts. A framework for impact evaluation in grant applications has been developed. In this framework, the criteria for evaluating impact statements focus on process-oriented impacts and that 'impacts' can be outputs, outcomes, and medium-term impacts.