What is this problem?

Our ability to solve problems in the world relies heavily on our ability to make high-quality decisions. We need to be able identify what problems to work on, to understand what factors contribute towards these problems, to predict which of our actions will have the desired outcomes, and to respond to feedback and change our minds.

There’s plenty of reason to think that our decision making competence is currently less-than-perfect, as psychology research over the past few decades has documented a whole host of cognitive biases affecting judgements and decisions.

For example, when we try to judge our chances of success, we focus too much on all the reasons why our case will be different from average: despite the fact most startups fail, most prospective entrepreneurs are convinced they will be the unusual case that succeeds.²

We’re also often overconfident in our predictions³ – it’s been argued that unwarranted confidence contributed to the explosion of the space shuttle Challenger, for example, where NASA overruled the safety concerns expressed by an engineer.⁴

Many of the most important problems in the world are incredibly complicated, requiring an understanding of complex interrelated systems, an ability to predict the outcome of different actions, and to balance competing considerations. That means there’s all the more room for errors in judgement to slip in. Even experts in political forecasting often do worse than simple actuarial predictions, when estimating the probabilities of events up to 5 years in the future.⁵ The organisations best placed to solve the world’s most important problems are also often highly bureaucratic, meaning that decision-makers face many constraints and competing incentives, not always aligned with better decision-making.

We think that improving the decision-making competence of key institutions may be particularly crucial, as the risks we face as a society are rapidly growing.

With technological developments in nuclear weapons, autonomous weapons, bioengineering, and artificial intelligence, our destructive power is quickly increasing. Crises resulting from war, malicious actors, or even accidents could claim billions of lives or more.⁶

It’s not clear that individual decision-making, or the structure of key institutions, has evolved at anything near the pace needed to manage these potential crises – our institutions and decision-making processes look pretty similar to those that failed in the first and second world wars, and yet the worst-case scenarios they need to mitigate are several orders of magnitude larger.

But there’s some good news. Research is beginning to focus on techniques to improve human judgement and decision-making. Researchers are studying how to improve our ability to make predictions about the future, how to better think probabilistically, and how to think about complex problems in a more structured way.

It seems like developing and applying strategies that improve human judgement and decision-making could be very valuable – particularly if focused on institutions working on particularly important problems, and combined with a thorough understanding of how such institutions operate.

Why work on this problem?

Improving decision-making could help us to solve almost all other problems

Taxes are pretty boring. So most people in the U.K. could be forgiven for not noticing when the British government’s Behavioural Insights Team (BIT) started running some simple randomised control trials (RCTs) to increase tax repayments in 2010. But these simple trials had a big effect – bringing forward over £210 million of extra revenue for the government in just a year.⁷

This first trial helped BIT to make the case for more evidence-based policy – now, RCTs are being used to improve the quality of decision-making across all policy areas. Seven years after the first tax trial, BIT have run over 300 RCTs across all areas of policy (including crime prevention, giving and social action, and counter-extremism).⁸

This is a good example of how improving judgement and decision-making improves our ability to solve almost all other problems. If we could improve the judgement of government officials facing high-stakes decisions – reducing their susceptibility to various biases, or developing better methods of aggregating expertise – this could have positive knock-on effects across a huge range of domains. For example, it could just as well improve our ability to avert threats like a nuclear crisis, as help us allocate scarce resources towards the most effective interventions in education and healthcare.

The potential for improvements in decision-making to impact a broad range of important issues means it’s a particularly good approach to focus on if you’re uncertain about what problems are most important.

There are several ways to make progress on improving decision-making

Research so far has made some progress identifying techniques that reliably improve judgements and decision-making, and at least where there is good evidence, there does seem to be growing interest in getting techniques implemented in practice.

Philip Tetlock’s work on forecasting,⁹ for example, has identified a number of ways to improve the accuracy of predictions on real-world events, which have been tested in large-scale RCTs. A separate research programme on prediction markets suggests getting people to “bet” on their predictions can increase the accuracy of political forecasts, corporate predictions, and statistical weather forecasts.¹⁰

(Incidentally we interviewed Tetlock about his research, when to trust experts and his career advice for people working on improving institutional decision-making on our podcast.)

With increasingly solid evidence for these effects, getting them practically implemented seems achievable – there is already interest in forecasting techniques in the intelligence community, for example.¹¹ One promising next step here would be to take findings that we have solid evidence for, and run smaller-scale pilots in more specific contexts (i.e. specific organisations or parts of government.)

For instance, people’s judgements can be dramatically overconfident, leading to poorly-informed decisions – unjustified confidence in Iraq’s possession of WMDs had dramatic consequences, as we’ve seen. Part of the problem here is that people aren’t very well calibrated – our brains don’t seem to have a very good intuitive sense of what it means so say something is, say, “70% likely”, and as a consequence, the statements we make with 70% confidence turn out to be true much less than 70% of the time.

But there’s some evidence that it’s possible to improve people’s calibration through training. Hubbard Decision Research have trained over 1,000 people in calibration and found that 80% of participants were ideally calibrated after five exercises.¹²

More research here – particularly larger-scale RCTs, and studies that look at the effects of calibration training on real-world judgements (rather than just trivia questions) – could help strengthen the case for implementing these techniques in practice.

Other techniques that might be promising include structured analytic techniques (SATs)¹³ for reducing biases in judgement, and the Delphi method¹⁴ for aggregating different viewpoints – we discuss these and other promising approaches in more detail later.

Work in this area is relatively neglected

Work on “improving decision-making” very broadly isn’t all that neglected. There are a lot of people, in both industry and academia, trying out different techniques to improve decision-making. The British government spends a lot of money on management consulting¹⁵, for example, and there are researchers working on questions related to decision-making at all major universities across a range of different disciplines: including psychology, economics, business, marketing, and political science.¹⁶ However, there seems to be very little work focused on rigorously testing different techniques to get strong evidence of effectiveness, or putting the best-proven techniques into practice in the most influential institutions.

For example, there has been a lot of work testing and using “scenario analysis” techniques, with thousands of papers discussing various methods of correcting for overconfidence by considering a wider variety of possible scenarios. But it wasn’t until 2005 that someone published an experiment actually testing whether scenario analysis improved prediction accuracy – and as it turned out, it didn’t.¹⁷ This illustrates how, without good evidence on the effectiveness of different techniques, a lot of well-intentioned effort on “improving decision making” might be wasted.

Improving decision-making also seems more neglected than other ways of trying to “improve the system”, such as education, suggesting this work is more effective. People often argue for investing in education, or for certain kinds of political reform, for similar reasons we’ve given here: because these things will help us better tackle all kinds of problems.

For instance, the US government spends around 4.6% of GDP on education (800 billion dollars),¹⁸ and in a survey of the top 100 US foundations by GiveWell US education accounted for 15% of spending, beaten only by healthcare.¹⁹ By contrast, there are no sources of government funding or charitable efforts explicitly directed at improving institutional decision-making processes in the ways we’ve discussed. By contrast, despite the potential importance of artificial intelligence in the 21st century, we could only identify a handful of people working on systematic methods to forecast its speed of development and likely impacts (we interviewed one of those researchers on our podcast).

What’s more, there’s reason to think that focusing on institutions directly might be a more effective way to improve decision-making than a broad approach to improved education, as it targets a smaller set of people who already have a lot of influence, and focuses more on institutional processes.

It’s worth considering why this work hasn’t received a lot of attention so far, and whether it might be neglected for good reason. One such reason might be if it really is incredibly difficult to get large institutions to adopt new practices – we discuss some of these potential barriers in the next section. In addition, the kinds of large-scale controlled trials needed to rigorously test techniques can be expensive and time-consuming.

However, it might also be the case that this work is neglected simply because not many people are motivated to do it – perhaps most academics don’t find this kind of research all that intellectually rewarding. This suggests that if you are well-placed and motivated to do this work, it could be particularly impactful to do so.

What are the major arguments against this being pressing?

Difficult to get change in practice

Perhaps the main concern with this area is that it’s not clear how easy it is to actually get better decision-making strategies implemented in practice – especially in bureaucratic organisations, and where incentives are not geared towards accuracy.

It’s often hard to get groups to implement practices that are costly to them in the short-term – requiring training, resources, and changes from the status quo – and only promise long-term or abstract benefits. For example, there’s been some resistance to getting prediction markets implemented in practice, as running prediction markets with ‘real money’ is probably an illegal form of gambling (and even if you could get around this, it’s legally complex).²⁰

However, these problems may be surmountable if we can find ways to show decision-makers that techniques will help them to achieve the objectives they care about.

For example, when the Behavioural Insights Team wanted to show that using RCTs could improve the quality of policy decisions, they initially focused on an area where it was relatively easy to show improvements on immediate outcomes that government cared about – saving money.

As we saw, BIT’s first project showed government that they could notably increase the number of people paying their taxes, saving the government hundreds of millions of pounds.²¹ Once they’d proven the effectiveness of their approach in this area, they were able to move into more complex policy areas with harder-to-measure outcomes, but where their impact could potentially be even higher, like education.²²

Recognising the difficulties of getting change in practice might also mean it’s especially valuable for people thinking about this issue to develop an in-depth understanding of how important groups and institutions operate, and the kinds of incentives and barriers they face. It seems plausible that overcoming bureaucratic barriers to better decision-making may be even more important than developing better techniques. This is something we plan to explore more in future, by talking to more experts about the most effective ways to change, or work around, the kinds of constraints that influential decision-makers face in practice.

You might want to work on a more specific problem

Suppose you think that climate change is the most important problem in the world today. You might believe that a huge part of why we’re failing to tackle climate change effectively is that people have a bias towards working on concrete, near-term problems over those that are more likely to affect future generations. And so you might consider doing research on how to overcome this bias, with the hope that you could make important institutions more likely to tackle climate change.

However, if you think the threat of climate change is especially pressing compared to other problems, this might not be the best way for you to make a difference – even if you discover something useful about reducing the bias to work on immediate problems, it might be very hard to get that implemented in a way that’s going to directly make a difference to climate change.

Instead, it’s likely better to focus your efforts on climate change more directly, for example working for a think tank doing research into the most effective ways to cut carbon emissions.

The advantage of broad interventions like decision-making is that they can be applied to a wide range of problems. The disadvantage of working in this area is it might be harder to target your efforts towards a specific problem. So if you think one specific problem is significantly more urgent than others, and you have an opportunity to work on that problem more directly, then it is likely more effective to do so.

There might be better ways to improve our ability to solve the world’s problems

One of the biggest arguments for working in this area is that if you can improve the productivity or judgement of people working on important problems, then this increases the effectiveness of everything they do to solve those problems.

But you might think there are better ways to increase the speed or effectiveness of work on the world’s most important problems.

For example, perhaps the biggest bottleneck on solving the world’s problems isn’t poor decision-making, but simply lack of information – people may not be working on the world’s biggest problems because they’re lacking crucial information about what those problems are. Being more rational won’t help them if they don’t have that information.

A lot of work on promoting effective altruism might fall in this category – giving people better information about the effectiveness of different causes, interventions, and careers.

What can you do in this area?

We can think of work in this area as falling into several broad categories:

1. More rigorously testing existing techniques that seem promising
2. Doing more fundamental research to identify new techniques
3. Fostering adoption of the best proven techniques in high impact areas
4. Directing more funding towards all of the above

All of these areas seem pressing and seem to have room to make immediate progress (we already know enough to start trying to implement better techniques, but stronger evidence will make adoption easier, for example.) This means that which area to focus on will depend quite a lot on your personal fit and opportunities – we discuss each in more detail below:

1. More rigorously testing existing techniques that seem promising

The idea here would be to take techniques that seem promising, but haven’t been rigorously tested yet, to try to get stronger evidence of where and whether they are effective. Some techniques or areas of research that fall in this category:

• Calibration training – improving the accuracy of probability judgements – has a reasonable amount of evidence suggesting it is effective. However, most calibration training focuses on trivia questions – testing whether this actually improves judgement in real-world scenarios could be promising, and help to get these techniques applied more widely.

• Structured Analytic Techniques (SATs) are a number of techniques developed for reducing cognitive biases in intelligence analyses. Examples of SATs including checking key assumptions, and challenging consensus views. These seem to be grounded in an understanding of the psychological literature, but few have been tested rigorously (i.e. with a control group and looking at the impact on accuracy.) It could be useful to select some of these techniques that look most promising, and try to test which are actually effective at improving real-world judgements.²³ It might be particularly interesting/useful to try to directly pitch some of these techniques against each other and compare their levels of success.

• Methods of aggregating expert judgements, including Roger Cooke’s Classical Model for Structured Expert Judgement, which scores different judgements according to their accuracy and informativeness and then uses these scores to combine them,²⁴ and the Delphi Method, a method for building consensus in groups, by using multiple iterations of questions to collect data from different group members.

Practically speaking, this probably means trying to get a position at a behavioural science/research lab interested in working on these kinds of questions and studies (which would very likely require getting a PhD in psychology or related area of behavioural science.) Some labs and researchers we know of whose research interests seem to fall in this area:

• Professor Philip Tetlock at the University of Pennsylvania, who has pioneered research in improving forecasting.
• Professor Robin Hanson, an economist at George Mason University who has researched and advocated for the use of prediction markets.
• Professor George Wright at Strathclyde University in Glasgow, a psychologist who does research into applying the Delphi method and scenario thinking to improve judgements.
• Stephen Coulthart at the University of Texas has done some research looking at the effectiveness of different Structured Analytic Techniques (SATs) in intelligence analysis.²⁵
• There are some groups focused on more applied decision-making research which may have money to run large field experiments, particularly in business schools, including: the Behavioural Insights Group at the Harvard Kennedy School; the Operations, Information and Decisions Group at the Wharton School, Pennsylvania; the Decision Sciences Group at Duke’s Fuqua School of Business; the Center for Decision Research at The University of Chicago Booth School of Business; the Behavioural Science Group at Warwick Business School.

There may also be some non-academic organisations with funding for, and interest in running, more rigorous tests of known decision-making techniques:

• IARPA is probably the biggest funder of research in this area right now, especially with a focus on improving high-level decisions.
• Consultancies with a behavioural science focus, such as the Behavioural Insights Team, may also have the funding and interest in doing this kind of research. These organisations generally focus on improving lots of small decisions, rather than on improving the quality of a few very important decisions, but they may do some work on the latter (e.g. as mentioned above, BIT have focused not just on small ‘nudges’ but also on improving the quality of policy decisions at a high level by promoting the use of RCTs.)²⁶

2. Doing more fundamental research to identify new techniques

You could also try to do more fundamental research; developing new techniques and approaches to improved judgement and decision-making, and then testing them. This is more pressing if you don’t think the existing techniques are very good.

One example of an open question in this area is: how do we judge “good reasoning” when we don’t have objective answers to a question? (i.e. when we can’t just judge answers/contributions based on whether they lead to accurate predictions or answers we know to be true.)²⁷ Two examples of current research programmes related to this question are IARPA’s Crowdsourcing Evidence, Argumentaion, Thinking and Evaluation (CREATE) programme and Philip Tetlock’s Making Conversations Smarter, Faster (MCSF) project, so you could try to get involved with one of the teams working on these projects.

One example of a project funded by the aforementioned CREATE programme is Bayesian Argumentation via Delphi (BARD), which is a collaboration between UCL, Birkbeck, and Monash universities. BARD uses causal Bayesian networks and automated Delphi analyses to help groups of analysts develop, improve and present their analyses. Note that there’s a considerable software development component to this project – suggesting that part of improving decision-making may involve not just developing psychological techniques, but also developing software and tools that can aid decision-making.

You could also work on a similar project independently if you are a researcher who can get funding for this, or if you’re thinking about doing a PhD try to work with academics in related areas. The researchers at Monash, UCL, and Birkbeck mentioned above are some we’re aware of, but there are likely many others.

The academics and institutions listed in section 1. (on testing existing techniques) might also be promising places to work if you’re interested in developing new decision-making techniques.

3. Fostering adoption of the best proven techniques in high-impact areas

Alternatively, you could focus more on implementing those techniques we currently think are most likely to improve collective decision making (such as the research on forecasting by Tetlock, prediction markets, or calibration training.)²⁸ If you think one specific problem is particularly important, you might prefer to focus on the implementation of techniques (rather than developing new ones), as this is easier to target towards specific areas.

As mentioned above, a large part of ‘fostering adoption’ might first require better understanding the practical constraints and incentives of different groups working on important problems, in order to understand what changes are likely to be feasible. For this reason, working in any of the organisations or groups listed below with the aim of better understanding the barriers they face, might be valuable even if you don’t expect to be in a position to change decision-making practices immediately.

These efforts might be particularly impactful if focused on organisations that control a large number of resources, or organisations working on important problems. Here are some examples of specific places that might be good to work if you want to do this:

• Any major government, perhaps especially in the following areas: (1) intelligence/national security and foreign policy, (2) parts of government working on technology policy (the Government Office for Science in the UK, for example, or the Office of Science and Technology Policy in the US), (3) development policy (DFID in the UK, USAID in the US), or (4) defense (the Department of Defense in the US, the Ministry of Defence in the UK). (We have upcoming career reviews on some of these paths.) Many parts of government are also now hiring for “behavioural science” specialist roles,²⁹ which might be a good option if you’re qualified for them (which generally means a masters/PhD in psychology or a related discipline, and ideally some policy experience.)
• Organisations that direct large amounts of money towards solving important world problems – such as foundations or grants agencies. e.g. the Gates Foundation, or the Open Philanthropy Project. See also our profile on working as a foundation grantmaker.
• Think tanks and academic institutions focused on questions about the long-run future of humanity, which require making judgements about speculative long-term scenarios, such as the Future of Humanity Institute in Oxford and the Centre for the Study of Existential Risk in Cambridge.

You could also try to test and implement improved decision-making techniques in a range of organisations as a consultant. Some specific organisations where you might be able to do this, or at least build up relevant experience, include:

• Working at a specialist “behavioural science” consultancy, such as the Behavioural Insights Team or Ideas42. Some successful academics in this field have also set up smaller consultancies – such as Hubbard Decision Research (which has worked extensively on calibration training).³⁰
• Good Judgement is an organisation founded on the basis of Tetlock’s successful research project on forecasting, and who now run training for individuals and organisations applying these findings to improve predictions.
• HyperMind, an organisation focused on wider adoption of prediction markets.
• Going into more general consultancy, with the aim of trying to specialise in behavioural science or decision-making – see our profile on management consulting for more details.

Finally, you could also try and advocate for the adoption of better practices across government and organisations, or for improved decision-making more generally – if you think you can get a good platform for doing so – working as a journalist, speaker, or perhaps as an academic working in this area.

Julia Galef is a good example of someone who has followed this kind of path. Julia worked as a freelance journalist before co-founding the Centre for Applied Rationality, co-hosted the podcast Rationally Speaking (which she still runs today) , and started a YouTube channel with tens of thousands of followers. She’s now writing a book on improving your own judgement, while running the Update Project, which focuses on helping decision makers improve their models of the world and resolve disagreements more productively. More broadly, she’s aiming to build an intellectual community of influential people across a range of different fields, who are genuinely trying to be truth-seeking and to resolve disagreements in a productive way. You can learn more about Julia’s career path by checking out our interview with her here.

4. Directing more funding towards research in this area

One challenge for all of the above areas is that it may be difficult to get funding for the kinds of work and research involved. Another approach in this area, therefore, might be to move a step backwards in the chain and try to direct more funding towards work in all of the aforementioned areas: developing, testing, and implementing better decision-making strategies.

The main place we know of that seems particularly interested in directing more funding towards improving decision-making research is IARPA in the US.³¹ Becoming a program manager at IARPA, if you’re a good fit and have ideas about areas of research that could do with more funding,is therefore a very promising opportunity. There’s also some chance the Open Philanthropy Project may invest more time/funds in exploring this area (they have previously funded some of Philip Tetlock’s work on forecasting). Otherwise you could try and work at any other large foundation with an interest in funding scientific research where there might be room to direct funds towards this area.

If you work at any of the organisations listed above, you might also try to advocate for more funds to be directed towards testing and implementing improved decision-making practices, even if you’re not in a position to do the work yourself.

If you don’t have the relevant background to do research or implement better practices yourself, but you think this is important and you’re in (or think you could work up to) an influential position in an important organisation, then you might be able to allocate more funding towards improving decision-making practices where you work (e.g. by funding tests, hiring someone who would be able to do this work.) If you’re in a position to do this, this could be even higher-impact than working somewhere like IARPA, where there already seems to be a lot of motivation to direct funding towards these problems.

We don’t currently think that there are many great direct donation opportunities in this area, and so this probably isn’t the best way to have an impact – at least not for relatively minor donors. If you’re a larger donor, though, you might consider funding academics to do the sort of research outlined in points 1 and 2, or even trying to set up an organisation to conduct and/or fund more of this kind of research.

Next steps

Learn more:

• Listen to our podcast episodes on forecasting with Philip Tetlock: Prof Tetlock on predicting catastrophes, why keep your politics secret, and when experts know more than you and Accurately predicting the future is central to absolutely everything. Professor Tetlock has spent 40 years studying how to do it better.
• We also recommend our podcast with: Mushtaq Khan on using institutional economics to predict effective government reforms
• For other angles, check out our podcast episode on voting reform with Aaron Hamlin, our podcast episode on Taiwan’s experiments with democracy with Audrey Tang and our podcast episode on misaligned incentives in international development with Karen Levy.
• Learn about Our World in Data’s strategy for making important information about the biggest issues out there available to decision makers everywhere.
• See a talk by Tyler John on ways governments can take into account the interests of future generations.
• And check out a talk from Max Stauffer on strategies to support better political decision-making.
• Read’s Lizka’s forum post, Disentangling “Improving Institutional Decision-Making.
• If you’re interested in doing research in this area, our career profile on valuable academic research gives more detail on that general career path.
• If you’re particularly interested in improving forecasting, you could sign up to the Hybrid Forecasting Competition (part of the HFC research program funded by IARPA) to participate and learn more – the program aims to combine statistical and human judgements to make forecasting systems as accurate as possible.
• If you think you might be better placed to get techniques implemented, or to direct funding, check out our profiles on policy-oriented civil service, and working as a foundation grantmaker.
• To learn more about existing research in this area, consider reading: (1) Philip Telock’s Superforecasting, (2) Robin Hanson’s work promoting prediction markets, (3) Some of the other research emerging from IARPA’s relevant grants, in particular: ACE, HFC, FUSE, and SHARP.
• See also the work of Andy Matuschak, Michael Nielsen and Bret Victor on "tools for thought", new mediums for science and engineering and "programmable attention". Also this blog post about their work, by a former Chief Advisor to the UK Prime Minister.

Take action:

• Consider getting a PhD in behavioural/decision science one of the groups mentioned above. See if you can find a supervisor and/or funding that would allow you to run large trials of existing techniques, or research new strategies for improved decision-making.
• If you already have the relevant background (e.g. a PhD, or policy experience), try to work in government (or a similarly influential organisation) where you might be able to either work on implementing better practices directly, or direct resources towards others who can do this work.