Charity evaluation is rarely straightforward. Many factors, within a charity’s control or outside of it, can influence the impact a charity has.

This blog post will highlight a case that illustrates how thinking through these factors can lead to surprising information that changes our understanding of a charity’s impact.

Summary

GiveWell recommended a grant to Results for Development (R4D) in May 2016 for its recently-launched program to increase access to pneumonia treatments for children in Tanzania. We thought this program was promising enough to potentially join our short list of GiveWell top charities once we had more information on its impact.

Expanded access to treatments is a factor in reducing child mortality from pneumonia, but not the only factor. We ultimately want to know not just whether more pneumonia treatments are available in Tanzania, but whether fewer children die of pneumonia as a result of R4D’s work. We expect the program to best achieve this impact if pneumonia patients visit health clinics with treatments in stock and are diagnosed and treated correctly.

We learned as we followed R4D’s work that there was limited information available on the accuracy of clinicians’ pneumonia diagnoses. We initially guessed that clinicians were diagnosing pneumonia accurately around 80 percent of the time. R4D collected data on diagnostic accuracy and we learned that the rate of accurate pneumonia diagnosis was actually 18 percent. This caused our estimate of the program’s impact to fall, though it remains in the range that we look for in potential top charities.

This finding highlights why it’s important to think through all of the factors along the path from a charity’s activities to its ultimate impact; if we had just considered whether more treatments were available, we would have missed this part of the story. We’re excited to continue following R4D’s work because of the role it has played in collecting this information to date and our expectation that it will continue collecting information that allows us to estimate its impact on the availability of pneumonia treatments across Tanzania. We expect to consider R4D as a potential future top charity.

In this post, we discuss:

• The background for GiveWell’s grant to R4D (More)
• Our plans for assessing the impact of R4D’s program (More)
• Approaches to measuring R4D’s impact (More)
• Lessons from this work (More)

Read More