Evidence and policy

Blog post:

Ten simple insights from Kat Smith and Paul Cairney synthesising the relationship between evidence and policy, with an introduction from Sean Innis.

The role of evidence in policy is oft discussed but little understood.  There is a powerful common view that evidence should determine public policy and much wringing of hands from people who feel government is ignoring the evidence.

To a degree this is fair. Policy should be grounded in a thorough understanding of the existing evidence. Too often, policy is made in ignorance of the evidence. But seeing a deterministic relationship between evidence and policy is far too simplistic.

In this recent article Kat Smith and Paul Cairney do a wonderful job of synthesising the relationship between evidence into ten simple insights. These insights resonate with me strongly and are valuable for both academics and policymakers alike.

Provocatively the authors suggest that there is no such thing as “the evidence”. Instead they see a world of competing insights based on varying assumptions, methodologies and problem definitions. This leaves evidence being “contestable and open to interpretation”.

As a policymaker this was always my experience. To take one example – every evaluation I have seen of a social policy intervention provided evidence of a distribution of consequent behaviours and outcomes. Even the most controversial and disliked of policies showed evidence of good outcomes for some and poor outcomes for others. Using this evidence necessarily becomes a matter of interpretation: on what basis do we say that the good outweighs the bad or vice versa?

Those that promote one interpretation of contestable evidence as definitive, create justifiable frustration within the policy community.

It is a frustration that goes both ways. Policymakers are also often guilty of seeking “the evidence” in a world where it does not exist. Academics well know that policy evidence collects and adjusts over time. As the authors suggest “refining policy-relevant evidence is frustrating but necessary”.

Policymakers often ask the understandable but often unfair question “what does the evidence say will happen if….” as if there is a single, definitive answer. This search for definition often results in a reliance on evidence that is incomplete or ill-suited for the purpose used. The use of modelling results sometimes falls into this category.

An abstraction of reality is taken to be a reflection of reality

Part of the tension in policymaking is captured by the authors’ sixth insight that “it can be strategically useful to present decisions as ‘evidence-based’, but policy is necessarily political”. This is undeniably true, but here I have a nuance. My experience is that it is valuable to decompose political decision-making a little further.

Political decisions draw from two broad rationales. The first relates to the principles which guide decision-making in a way that complements the use of evidence. Principles are by nature an expression of ideology. These principles can be shared across political viewpoints or they can be fiercely contested.  It is no surprise that policies which reflect broadly shared principles tend to endure the longest.

The second rationale draws from the conduct of democratic politics. Here decisions either reflect a desire to gain political advantage over an opponent, or manage the practical realities of gaining enough support for a policy to be accepted (eg passed by parliament).

As a policymaker, my experience was that there was always great value in seeking to make transparent both the evidence and the principles being used to inform decisions. Where this combination was strong, the politics of advantage and pragmatism tended to play a lesser role in decisions. This was not universally true but true enough to be valuable.

The bottom line, and the important thing for both academics and policymakers to understand, is that evidence never stands alone. The more understanding we can bring to how evidence actually supports policy, the better policy-making will be in the future.

Evidence never stands alone

I highly commend Kat Smith and Paul Cairney’s article to you.  

By Sean Innis.
Sean Innis is inaugural Director of the Public Policy and Societal Impact Hub at the Australian National University. Sean is a former senior public servant with more than 25 years public policy experience, including as Special Adviser to Australia’s independent Productivity Commission in 2016 and 2017 and senior executive positions in the Department of the Prime Minister and Cabinet and the Department of Social Services.

If these ideas spark a reaction, and if you can manage it, please consider posting a comment on the article in the i2insights.org blog, to promote discussion.

Kat Smith PhD is a professor of public health policy at the University of Strathclyde in the UK. Her key research interests are analysing who influences policies impacting on public health and how, tackling health inequalities, and studying innovation in health taxes.

Paul Cairney PhD is a professor of politics and public policy at the University of Stirling in the UK. His key research interests are in policy processes, including the idea of ‘evidence based policymaking:’ https://paulcairney.wordpress.com/ebpm/.

Leave a Reply