When intuition isn't good enough

Recently I was at a talk given by Rachel Glennerster of the Abdul Latif Jameel Poverty

Action Lab (JPAL).  In speaking about cost-effectiveness, she gave examples from

several different education interventions in Africa:

1) merit scholarships for girls

2) free primary school uniforms

3) providing information to parents on how schooling increases income

4) deworming through primary schools

All the interventions sounded pretty good to me. I could see how all of them might increase the number of years children spend in school.

But JPAL compared randomized controlled trials of each intervention, and when they looked at the additional years of education you get for the cost, these are the results:

Source: JPAL

This isn't perfect information: the studies are from different countries, and as with any studies, there's some uncertainty. But it's a lot better than relying on guesswork.

In retrospect, it makes sense that parents would prioritize schooling if they know more about education's economic benefits to their children. But I can make up just-so stories for why any of these would be the most effective. You could have presented me with any of those interventions, told me it was a great method to get children more schooling, and I would have found it believable.

(In fact, the Madagascar study tested whether it works better to give parents statistics about children's expected earnings or to have "role models" speak to parents about how education benefited them. The role model intervention is more common, based on the theory that impoverished parents with poor literacy aren't able to understand statistics. But the study indicated that they are able to understand the information given, and that it actually works better than having role models speak to them.)

These studies go beyond asking "Does it work?" That's a good starting place — after all, some interventions don't work at all — but it's not enough. Everyone from small donors to government policymakers needs to know about cost-effectiveness. Unless we have infinite money, we need to know where our money will go the farthest so we can start there. If we have $100 to give, there's a big difference between buying school uniforms (which we expect to result in less than one year of additional education) and providing information to parents (which we expect to result in over twenty years of education!)

This is why research is important. I'm glad there are people out there like JPAL getting more information so we're not just guessing.  

You can hear Rachel Glennerster's talk and others from the Good Done Right conference here. Apologies for the occasional baby sounds in the background, which are my daughter.

Too narrow?

I'm always interested to see what people criticize about the effective altruism movement. One critique I noticed lately is that we're too narrowly focused on a few interventions and instantly reject anything that doesn't have a GiveWell stamp of approval. There's some truth to that.

A related critique is, “If everyone focused so narrowly, virtually nothing would get done – no art, no civic projects, just mosquito nets all the time."

(The art thing is especially touchy. Every time someone from the effective altruist movement suggests that symphonies and museums might not be the best charitable causes, people freak out about how we hate the arts.  I'd like to invite them to see Robbie's photography, hear Ray's music, go to one of Catriona's Shakespeare productions, or talk architecture with Toby.)

While I'm thrilled to see more people interested in effectiveness, I don't see such a stampede to effective causes that I think we're about to fund all the best giving opportunities.  If that ever happens, I have no doubt that most effective altruist types will move on to whatever cause seems next best.  For example, smallpox was a huge problem well into the 20th century.  But now that vaccination has been so successful, people have moved on to other causes.

Aside from the fact that effective altruists will move on to next-best projects after the best ones are taken care of, I'm not concerned that a narrow focus will mean that all non-optimal causes get neglected. Even those of us who focus on finding effective causes don't spend all our time and money on them.  I spend my days as a mental health clinician for prisoners. I am about to start the (possibly even more emotionally intense) work of parenting.  I don't do those things because I think they're the most effective ways to improve the world – I do them because I want to, because I find them satisfying.  But at the end of the day, I want to give my earnings to the most effective organization I can find. And it annoys me when people categorize me as some kind of heartless utility-bot because of that.

If you're upset that effective altruist types don't seem concerned about some issue in your local community, the problem might not be that we're too narrow-minded.  In fact, one of us may well have a day job in that field, or have a special place in our heart for spending our Saturdays working against water pollution or homelessness or whatever.

But that's not the same as saying, "My pet cause is so important and has so much opportunity for change that everyone should drop what they're doing and work on this."  I have a special place in my heart for refugee services, and I've volunteered in that field for years.  But I don't think it's the best thing everyone should be doing, and I'm not saying it should get top billing as an effective cause.

Meanwhile, I think interventions like deworming, nutritional supplementation, and malaria prevention really do deserve top billing.  Some of my money and time will still go to other areas, but I will prioritize the causes that have been demonstrated most effective.

Sometimes it's good to narrow things down.

Rooting for your home team

There's a gap between charity evaluators and charity workers. Look at any article on effective charity, and you're likely to see responses from NGO staff voicing their frustration with the evaluation methods:

"You don't understand how hard it is to persuade donors to fund evaluation."

"We're reaching a population that no one else is reaching."

"The type of work we do just isn't measurable by randomized controlled trials."

"It's hard for really small organizations to get noticed."

The thing that worries me about these responses is that it's hard to be objective about your own project. I've been there. When I worked for a nonprofit doing global development and relief work, I was rooting for them. I knew their values, I knew their staff, and I loved both. You see this in devoted donors, too - once you've helped with a project, you want to see it continue. It's only natural, once you feel like part of the group.

But which charity you support is not the point. The point is to help people who need help. Right? That's why we're doing this, remember?

The flaws in any evaluation (and there will be flaws) are bad not because they shortchange charities, but because they shortchange the people who should be getting better help. The question we should be asking is not, "Is the evaluation fair to these organizations?" but "How can donors help the most?"

My home team isn't in an office in Boston. My home team is in a village somewhere. I'm rooting for them.