The Center for Global Development churns out any number of new ideas and energetically hawks them round northern 
governments and multilaterals: the benefits of migration, oil for cash, the Commitment to Development Index and many more (check out the Initiatives tab on their homepage). In recent years, Cash on Delivery aid has been one of their top products, and a number of donors have started to pick up the idea (although just to keep us on our toes, its name keeps changing – to ‘Results-Based Aid’, or ‘Payment by Results’. Apparently this is because what’s happening so far doesn’t meet the full set of COD criteria set out in CGD’s book).
According to the CGD, ‘Under COD Aid, donors would pay for measurable and verifiable progress on specific outcomes, such as $100 dollars for every child above baseline expectations who completes primary school and takes a test.’
The appeal of this approach is that it leaves governments and other recipients to find their own way to achieve good things, based on local contexts, rather than prescribing universal ‘best practice’ that often fails. So it fits well with other currents of thought on complexity and systems, (covered ad nauseam on this blog).
But now CGD researchers Bill Savedoff and Rita Perakis have stepped back and taken a long hard look at where CoD has got to thus far. I attended a discussion of the resulting paper “Does Results-Based Aid Change Anything?” last week, and I think it’s fair to say they haven’t exactly been blown away by what they’ve found.
Firstly, donors have been overclaiming: According to Rita and Bill’s blogpost on their paper ‘instead of a revolution in aid, we found a cautious adaptation of traditional programme approaches’.
They found very few programs that actually pay governments for outcomes (four to be precise, although the numbers have risen since they did their trawl). They were GAVI’s ISS (Immunization Services Support) programme; the Amazon Fund (a deforestation programme in Brazil, funded by Norway); an Ethiopian secondary education programme, funded by DFID) and Salud Mesoamerica 2015 (regional healthcare, 8 countries in Central America, funded by the IADB).
The authors paper identifies four theories for how RBA programmes are supposed to work:
Pecuniary interests. Countries will change their priorities because they need the money.
Attention. Because funds are linked to outcomes, politicians and bureaucrats will pay more attention to results and manage things differently than they would otherwise.
Accountability. RBA agreements make outcomes visible to citizens in funding and receiving countries, allowing them to hold their governments accountable for performance.
Recipient discretion. By linking payments to outcomes rather than inputs, funders give recipients wider latitude to design and implement strategies of their own making.
Out of the 4 case studies, the authors found that 3 were just about ‘attention’ – getting decision makers to think more about
outcomes. Only the Amazon Fund went further, including some elements of accountability and recipient discretion, and tellingly, that Fund did not come out of the standard aid agency processes. It was proposed by Brazil in 2007 and picked up by climate change staff in the Norwegian government rather than by NORAD.
Elsewhere, the researchers found a ‘Striking lack of transparency on the results of what were actually quite simple programmes’ and thus no impact on accountability (you can’t hold governments accountable if you have no information on what they are doing).
On the plus side ‘the typical concerns with RBA did not materialize, such as corruption, distortions of incentives, or the sacrifice of long goals’, although that’s hardly surprising given that the programmes are such a timid step towards RBA.
The discussion at CGD highlighted a few other issues:
Time horizon: The standard 3 year programme doesn’t give you time to invest upfront in innovation, so if you need to show results quickly, you play it safe. RBA needs to go to 5 year programmes or longer if it is genuinely interested in promoting innovation.
Sustainability: The Brazilian deforestation study got me thinking. Whether or not your RBA project is responsible, if it coincides with a downswing in the rate of deforestation, the Brazilian government gets lots of cash, and that’s great. But when the project ends, if an upswing takes place, and the trees get chopped down and burnt after all, the net effect on the amount of CO2 in the air is the same (Bill argues that a delay at least buys time to sort out a carbon transition, but that seems pretty thin). So maybe RBA works better for largely irreversible changes (eg educating kids) than for reversible ones?
Which leads on to the link between RBA and institutional strengthening. In the long run, only strong domestic institutions are going to deliver sustainable progress on health, education, deforestation etc, but the link between RBA and institutional strengthening seems pretty tenuous (you may need better institutions to get those short term results). Mind you the whole idea that outsiders can strengthen domestic institutions is pretty debateable – so the question of whether RBA is more/ less likely than traditional aid approaches to encourage stronger institutions is both open and very important.
Finally, this paper is only about aid to governments, and deliberately excludes using results-based performance contracts with other recipients such as NGOs. A BOND survey of NGOs about their experiences to date suggests that version of RBA is pretty disastrous.
And here’s CGD boss Nancy Birdsall explaining COD in ten minutes
Thanks for (yet another) great summary, Duncan.
Interesting study on, “Results-Based Financing Schemes in WASH” here http://bit.ly/1CcI4Vy (Finds PBR is effective, but not enough evidence to say whether it’s efficient or sustainable)
Blogs from interviews with a Vietnamese NGO on their PBR sanitation program, which, ref the Bond report, is explicitly aimed at the poorest sections of the communities and, I was persuaded, where PBR generated innovation http://www.euforicservices.com/search/label/oba
If Northern governments believe RBA delivers better outcomes, this should be thoroughly tested in their own health, education and forestry etc ministries before being rolled out in the South. Given challenges with institutional capacity and the immense obstacles to delivery in so many recipient contexts, there should be proof of concept in conditions that are amendable before it enters the complexity of development. I’d be interested to know of evidence of RBA type approaches in advanced economies if it’s out there.
Good point Ross, One of the comments at the seminar was on this – the lack of cross fertilization between experiences in countries such as UK and US, and those getting RBA aid programmes
Thanks, Duncan. Helps add a twist to my thinking which has focused more on the incentives created by performance based contracting (http://goo.gl/hkhJHK, http://goo.gl/H0WXc4).
Couple of points for me. One is that, in reference to those contracting arrangements, I do think – and have seen – that external support can lead to institutional strengthening. That’s the M4P raison d’etre, but that’s not the focus here.
The question I have, then, is whether such COD structures simply amount to direct budgetary support? In relation to government programming, aid contributions are often minimal, when we’re talking about bringing about systemic changes like improving education. Will governments actually change their policies based on aid inputs? Or will the aid simply pay for the changes that happen if they happen – which might be due to existing government policy, other donor programmes, exogenous factors etc. Where’s the additionality? Where it has ‘worked’ as you state above, it is simply in delivering small scale change directly delivered. If you’re just going to pay for some children to be educated or to protect some trees then I don’t think using the government as your delivery agent has any greater impact that doing it yourself as a donor. In the absence of a systemic change, it simply direct delivery in another guise, with no discernible goal of sustainable or large-scale change.
A second point is for Ross (above). I see one problem is that Northern governments have tried these schemes in their own countries (often for ideological/political reasons of new public management as opposed to any evidence of efficacy). In trying to export these schemes to less well functioning systems, the problems with them are exacerbated and new ones are created. Look at PPPs in health. In the UK they’ve deferred problems in search of improving stats. This may be short or even medium term but the longer it goes on for the more the system is undermined and the hole gets deeper. Transfer this to an African context and departmental budgets with far less wriggle room have been hollowed out by the skewed incentives created by PBR.
I don’t see how COD approaches with fixed threshold indicators can avoid creating perverse incentives for public service providers – whether in a developed or developing context (Campbell’s law is universal I would argue). The case study on education in Ethiopia illustrates the point; to avoid the ‘teach the test’ effects the threshold indicator opted for was students who simply sit the exam (not passing it). By avoiding one perverse incentive (teaching the test) a new one is created as it remains unclear how making more students take the exam reflects actual improvements in education. Verification of even this ‘simple’ indicator is problematic (see p. 32 of the paper).
Apart from this, the long- term institutional growth of any public service system can become skewed if it is geared towards external incentives rather than internal reform. As for the claim that COD fosters ‘accountability toward constituents’, the paper is rather thin on this, stating that the programmes reviewed were not transparent enough to have effects. Besides the growing evidence that transparency does not necessarily generate accountability, it’s unclear how COD approaches on themselves would incentivize greater responsiveness towards citizens rather than (even greater) responsiveness towards external donors. If there is any literature on this I would be happy to know of it…
Duncan,
Thank you for this fair synopsis of my colleagues Bill and Rita’s insightful new report on results-based aid.
I’d like to note that your concern about the sustainability of forest conservation programs could be applied equally to climate actions in any sector (energy, transportation, agriculture…).
You write: “But when the [forest conservation] project ends, if an upswing takes place, and the trees get chopped down and burnt after all, the net effect on the amount of CO2 in the air is the same”
You could as easily have written: “But when the solar panel delivery project ends, if an upswing takes place, and the coal gets dug up and burnt after all, the net effect on the amount of CO2 in the air is the same”
Just as renewable energy production doesn’t truly prevent fossil fuels from being burnt later, forest conservation measures don’t prevent trees from being burnt later. But in both cases all of us who enjoy a safe and stable climate benefit from lengthening the delay.
Between 2004-2014 Brazil decreased emissions from Amazon deforestation by nearly 80%, making it the country that has reduced its emissions by more tons of carbon dioxide than any other country on Earth. Meanwhile it grew its soy and cattle production. The policies it put in place to achieve this transformation–including legal recognition of indigenous territories, establishment of protected areas, increased law enforcement efforts triggered by satellite alerts, and withholding of credit from high-deforestation municipalities–have as good a chance of being durable as any. My colleague Frances Seymour has written more on the outlook for Brazil’s reforms here: http://www.cgdev.org/blog/can-brazil-stay-course-reducing-deforestation
Jonah
Research Fellow, Center for Global Development
This came out yesterday and is highly relevant…
http://onlinelibrary.wiley.com/doi/10.1111/dpr.12115/abstract?campaign=woletoc
well it might be if it wasn’t gated Ben! Any Open Access version available?