What is wrong with evidence based policy, and how can it be improved?. (August 2017)

• Record Type: Journal Article
• Authors: Saltelli, Andrea; Giampietro, Mario
• Is Part Of: Futures. Volume 91(2017)
• Page Start: 62
• Page End: 71
• Publication Date: 2017-08
• Subjects: Evidence based policy -- Science for governance -- Science and technology studies -- STS -- Post-normal science -- PNS -- Quantitative story telling; Economic forecasting -- Periodicals; Technological forecasting -- Periodicals; Economic policy -- Periodicals; Prévision économique -- Périodiques; Prévision technologique -- Périodiques; Politique économique -- Périodiques; Economic forecasting; Economic policy; Technological forecasting; Periodicals; Electronic journals; 330.0112
• DOI: 10.1016/j.futures.2016.11.012
• Languages: English
• ISSNs: 0016-3287

Abstract

Highlights: There is a crisis of science's governance forcing to reconsider evidence based policy as it is being practiced at present. The closure of any issue in a pre-established frame used for quantification may correspond to normative and political stances. The use of mathematical modelling and indicators conveys a spurious impression of precision, prediction and control. Better styles of evidence based policy should flag the existence of 'uncomfortable knowledge' usually avoided in policy discussions. We suggest a strategy – Quantitative storytelling – to opening the space of possible narratives and control their quality . Abstract: The present crisis of science's governance, affecting science's reproducibility, scientific peer review and science's integrity, offers a chance to reconsider evidence based policy as it is being practiced at present. Current evidence based policy exercises entail forms of quantification – often in the form of risk analysis or cost benefit analyses – which aim to optimize one among a set of policy options corresponding to a generally single framing of the issue under consideration. More cogently the deepening of the analysis corresponding to a single view of what the problem is has the effect of distracting from what could be alternative readings. When using evidence based policy those alternative frames become a kind of 'uncomfortable knowledge' which is de facto removed from the policy discourse. All the more so when the analysis is supported by extensive mathematical modelling. Thus evidence based policy may result in a dramatic simplification of the available perceptions, in flawed policy prescriptions and in the neglect of other relevant world views of legitimate stakeholders. This use of scientific method ultimately generates – rather than resolving – controversies and erodes the institutional trust of the involved actors. We suggest an alternative approach – which we term quantitative story-telling – which encourages a major effort in the pre-analytic, pre-quantitative phase of the analysis as to map a socially robust universe of possible frames, which represent different lenses through which to perceive what the problem is. This is followed by an analysis where the emphasis in not on confirmatory checks or system optimization but – the opposite – on an attempt to refute the frames if these violate constraints of feasibility (compatibility with processes outside human control); viability (compatibility with processes under human control), and desirability (compatibility with a plurality of normative considerations relevant to the system's actors).