Better results for less money: AI and synthesising knowledge

In most areas of life, we have become used to knowing about things previous generations could only wonder at. In just the past century it has become normal for us to see the clouds from above, intact bodies from the inside, and visualise the structure of a whole language.

Our lives are better because we have this new knowledge. Understanding the clouds using weather data, collected by land, sea, sky, and even space-based monitoring, helps us plan for how the weather can both nurture and threaten human life. Understanding the inside of our bodies, using scans and ever more complex genetic technology, helps us diagnose and treat illnesses less invasively, less painfully, and more successfully. Understanding language using statistics gave us machine translation, and now helps bring people together across the world.

Artificial intelligence (AI) is helping us gather and make sense of the vast amounts of data and knowledge generated in all these fields and beyond. AI helps make new knowledge possible; more excitingly, AI can also tackle the hard challenge of making new knowledge useful. Many of the ways our own lives are better than our parents come from new knowledge, and there are opportunities for AI to amplify the knowledge production process.

Policymaking is missing out

You might expect policymaking to have advanced as dramatically in the past century as fields such as healthcare, meteorology or linguistics. Sadly, it has not. A minister walking into the Department for Work and Pensions (DWP) today will be relying on knowledge not so different from that which William Beveridge relied on in the 1940s, when he wrote his eponymous report that led to the creation of the welfare state. The tables in the Beveridge report will be recognisable to any regular reader of DWP statistics releases, even though faster, more nuanced, and more localised knowledge should by now be the norm. However, the bigger risk is that we still often turn to single sources of knowledge (predictably often from one ‘great man’ such as Beveridge) when reviewing the evidence and exploring potential policy solutions.

This should both surprise and worry us in equal measure. The role of policymaking has no-doubt transformed: policymakers now do more, spend more, and touch people’s lives more often, in more fundamental ways.

However, a century ago, government spending as a share of GDP was 25%. This has since almost doubled; in 2022-23, government spending was around 45% of GDP. That extra spending reflects new government responsibilities, such as the NHS, social security, and social housing. It also accounts for the increasing scope and complexity of age-old responsibilities, for example the police now tackling cross-border cybercrime, as well as patrolling local streets. On top of all that, there are about 25 million more people in the UK than 100 years ago.

Policymakers have a harder job than ever. So, the government leader or senior official arriving to their new office committed to improving people’s lives does deserve some sympathy. Yet, no amount of energy put into making the best out of a limited budget will get you far if you cannot see where to go. It is time they got the knowledge they need to have a fighting chance at making progress.

Providing knowledge to transform policymaking

There is a better way to support policymakers to make a difference. It is known as evidence synthesis. Manually synthesising evidence requires expert teams to filter, assess, and summarise tens of thousands of pieces of research and other evidence, summing up what we know and what we still need to know. High-quality evidence synthesis is objective, transparent, and reproducible: the standards it follows are set out in a protocol published before the review process begins, and the data and decisions are available for reuse and reanalysis.

Evidence synthesis is used to good effect in some areas of policymaking already. The Campbell Collaboration’s Crime and Justice Group has worked with a group of governments to synthesise the evidence on countering violent extremism to inform criminal justice policies, to reduce crime and protect people from violence, helping to save lives. The UK’s What Works Centres have been early adopters in using evidence synthesis within public policy, with the toolkits published by the Youth Endowment Foundation, the Youth Futures Fund, and the Education Endowment Foundation. Many young people’s lives are better because of this work as interventions that do not make a positive difference are replaced by those that can, and the limited money available is used more effectively.

However, while evidence synthesis has helped to transform fields such as medicine, it is underused in public policymaking. This is mostly due to the fact that evidence synthesis projects using manual approaches take several years to complete; they are hard, slow and expensive. Most often such timescales are too slow-moving to be compatible with the policymaking process.

Experts in the field estimate that AI could speed up the different stages of the synthesis process by 30-90%. Instead of a policymaker completing a systematic review, before watching it grow outdated as new research emerges, AI tools could automatically identify new evidence and quickly update what we know. This is known as living evidence synthesis. Using AI tools well, in combination with human experts, now offers realistic possibilities of enabling every policymaker to discover collated bodies of knowledge about how to solve complex social problems, as naturally as we now search online.

Systematic, comprehensive, reusable bodies of knowledge

It is time to invest in AI so that evidence synthesis can become a routine tool for policymakers.

The transformative power of systematic, comprehensive, and reusable bodies of knowledge is undeniable. We use a dictionary without a thought, although dictionaries are among the biggest social science projects in history. They sum up vast bodies of empirical knowledge in a single line for daily use, in longer essays for research use, and in machine readable forms that enable the functioning of tools like spellcheckers, which we now take for granted. In other areas, extraordinary progress and scientific breakthroughs have been built on comprehensive knowledge bases, from the periodic table to the human genome project.

We also know that we are not making the progress we need on a whole range of social problems. Millions of people suffer - not because government lacks the will, but because it does not know how to achieve its goals. Hundreds of years ago, the British political philosopher, Sir Francis Bacon, wrote that knowledge is power. It is just as true that ignorance is impotence.

The Global Commission on Evidence to Address Societal Challenges has called for governments to invest in 100 evidence synthesis projects that cover the whole of public policy, using AI to produce them faster and cheaper, and keep them continuously updated. Like the Beveridge report, this investment would benefit generations of people to come.