Evaluation of Givewell Blogs

GiveWell’s money moved in 2020

7 months 3 weeks ago

2020 was another year of tremendous growth. GiveWell donors contributed over $240 million to our recommended charities (our "2020 money moved"), a 60% increase from the approximately $150 million we directed in 2019. This is part of an exciting, long-term trend. Just a decade ago, in 2010, GiveWell’s total money moved was $1.5 million.

We believe these donations will save tens of thousands of lives and benefit many others. This incredible impact would not be possible without the continued support and generosity of our donors. While our research enables us to identify and recommend highly cost-effective giving opportunities, our donors are responsible for turning those recommendations into real change for some of the poorest individuals in the world.

Read More

The post GiveWell’s money moved in 2020 appeared first on The GiveWell Blog.

Robin Dey

Early signs show that you gave more in 2020 than 2019—thank you!

1 year 1 month ago

Our donor community appears to have given significantly more in 2020 than 2019, according to early data on donations we processed.

Growth was strong relative to previous years—even 2019, which also had strong growth—and across many different dimensions. Overall, donations to GiveWell more than doubled in 2020.

We estimate that these donations will collectively save more than 12,000 lives; provide over 2 million deworming treatments to children, leading to an approximate increase in that group's lifetime earnings of more than $21 million; and deliver almost 3,000 cash transfers to low-income households. For simplicity, the impact estimates in this paragraph exclude some donation types, and so don't represent the full impact of donations to GiveWell in 2020.[1]

Read More

The post Early signs show that you gave more in 2020 than 2019—thank you! appeared first on The GiveWell Blog.

Catherine Hollander

GiveWell’s money moved in 2019

1 year 6 months ago

GiveWell donors contributed over $150 million to our recommended charities in 2019. This is the first year in our 13-year history we have reached this remarkable milestone.

While it is easy to celebrate this large headline figure, we should not lose sight of the transformative impact this number represents, nor should we overlook the compassion and generosity of the donors behind these dollars. We estimate that donations made to our top charities in 2019 will:

  • Avert the deaths of between 19,000 and 32,000 people,
  • Provide parasitic worm treatments to over 12 million children, and
  • Deliver unconditional cash transfers to over 14,000 families.[1]

Read More

The post GiveWell’s money moved in 2019 appeared first on The GiveWell Blog.

Robin Dey

GiveWell’s plans for 2020

2 years 1 month ago

Each spring, we share our plans for the year. Here, we highlight the work we plan to do in 2020 that is most likely to help us realize our mission of identifying and directing funding to highly cost-effective giving opportunities.((This post does not include a complete accounting of everything we plan to do in 2020. In particular, it does not include work aimed at primarily internal-facing results, such as improvements to internal staff communications.)) We focus on three projects:

  1. Expanding into new areas of research.
  2. Searching for new, cost-effective funding opportunities in our traditional research areas.
  3. Building our donor community.

Sharing our annual plans and publicly reflecting back on them a year later is our typical practice. This year, of course, is atypical. The plans we laid out internally at the beginning of the year have been disrupted by the COVID-19 pandemic. The plans we share in this post take the pandemic into account, but we are more uncertain than usual about what will happen in 2020. We expect that much of our work will go forward as anticipated, but we will be flexible if there are unforeseen disruptions or changes to our research agenda that result from the pandemic.

Read More

The post GiveWell’s plans for 2020 appeared first on The GiveWell Blog.

Catherine Hollander

Reflecting on our progress in 2019

2 years 1 month ago

GiveWell grew significantly in 2019. We hired 13 full-time staff members, bringing our total size to 37, and expanded our ability to take on new projects across domains. We feel positioned to do more and better work going forward as a result.

We see a strong indication that the amount of funding we directed to our recommended charities increased last year, too. While we haven't reconciled all giving from 2019, the value of donations we processed increased by about 30% in 2019.

We're proud of what we accomplished in 2019. We also fell short of some goals last year. Most notably, we failed to make as much progress as we planned in researching new areas of global health and poverty alleviation.

This blog post provides a brief look at our key successes and failures last year. A more detailed accounting of how our progress in 2019 compared to the goals we set is available on this page.

Read More

The post Reflecting on our progress in 2019 appeared first on The GiveWell Blog.

Catherine Hollander

How did we do in 2019? A preliminary look at our growth.

2 years 4 months ago

We see an early indication that GiveWell continued its trajectory of robust donor growth last year. The total value of donations processed by GiveWell increased 30% in 2019.[1]

We're sharing this data now because we believe it is an informative early update about our growth last year. However, GiveWell-processed donations don't tell the full story of our impact. Many donors who rely on our research give via our partner organizations or directly to our top charities. Their gifts account for the majority of donations due to our work and are not processed by GiveWell. Information about these gifts is time-consuming to gather and has usually led us to release our metrics data many months after the end of the year. We plan to release a complete 2019 metrics report and assessment of our impact, including donations not processed by GiveWell, later this year.

Here's what we know so far, based on the nearly complete information we have about donations we processed:

  • We processed $54.1 million in donations in 2019. Sixty-five percent of this amount was restricted to our recommended charities and 35% was unrestricted, which we may use to support GiveWell's operations.
  • Support from donors giving $10,000 to $100,000 comprised the largest proportion of our growth (35%).[2]
  • Returning donors who gave more than last year made up 75% of our growth in funds donated (excluding anonymous donations).[3]
  • We believe that the majority of our growth was organic and would have occurred without any outreach and marketing efforts from GiveWell, although we can attribute some to specific outreach and marketing initiatives.[4]

We're encouraged by this growth and excited to write about it. We also discuss below some ways that GiveWell-processed donations could be a misleading indicator of our overall impact.

Read More

The post How did we do in 2019? A preliminary look at our growth. appeared first on The GiveWell Blog.

Catherine Hollander

GiveWell’s money moved and web traffic in 2018

2 years 9 months ago

GiveWell is dedicated to finding outstanding giving opportunities and publishing the full details of our analysis. In addition to evaluations of other charities, we publish substantial evaluation of our own work. This post lays out highlights from our 2018 metrics report, which reviews what we know about how our research impacted donors. Please note:

  • We report on “metrics years” that run from February through January; for example, our 2018 data cover February 1, 2018 through January 31, 2019.
  • In an effort to present a more comprehensive measure of our influence on charitable giving, this year's metrics report includes GiveWell Incubation Grants in our headline "money moved" figure. In previous reports we have excluded Incubation Grants from this figure.

Summary of influence: In 2018, GiveWell influenced charitable giving in several ways. The following table summarizes our understanding of this influence.

Headline money moved: In 2018, we tracked $141 million in money moved to our recommended charities and via our Incubation Grants program. Our money moved only includes donations that we are confident were influenced by our recommendations.

Read More

The post GiveWell’s money moved and web traffic in 2018 appeared first on The GiveWell Blog.

Devin Jacob

GiveWell’s plans for 2019

3 years 1 month ago

Our top priorities this year support our goals to (a) increase the impact per dollar of the funds we direct and (b) increase our money moved. In 2019, we are focused on:

  • Building research capacity. (More)
  • Experimenting with approaches to outreach to find ones that we can scalably use to drive additional money moved. (More)
  • Exploring new areas of research. (More)
  • Improving GiveWell's organizational strength. (More)
  • Ongoing research. (More)

Read More

The post GiveWell’s plans for 2019 appeared first on The GiveWell Blog.

Catherine Hollander

Review of GiveWell’s work in 2018

3 years 1 month ago

2018 was a successful year for GiveWell. We achieved most of our goals and our money moved (donations made to our recommended charities due our research) increased significantly.

Each year, we look back at the goals we set the previous year and reflect on how our progress compared to our expectations.

This post will briefly discuss our key achievements and failures in 2018. We describe in detail our progress on the goals we outlined in 2018 here.

In 2018, we:

  • Directed an estimated $65 million in donations to our top charities, not including the contributions of Good Ventures, a large foundation with which we work closely.
  • Added senior hires in operations and outreach: a Director of Operations (Whitney Shinkle) and Head of Growth (Ben Bateman). We expect Whitney and Ben to make major contributions to our work in these domains.
  • Continued to improve and expand our core research product, completing new intervention reports, deepening our analysis for several key inputs into our cost-effectiveness model, and providing more transparent explanations for how we decided to allocate funds between top charities.

Read More

The post Review of GiveWell’s work in 2018 appeared first on The GiveWell Blog.

Catherine Hollander

How GiveWell’s research is evolving

3 years 4 months ago

To date, most of GiveWell's research capacity has focused on finding the most impactful programs among those whose results can be rigorously measured. This work has led us to recommend, and direct several hundred million dollars to, charities improving health, saving lives, and increasing income in low-income countries.

One of the most important reasons we have focused on programs where robust measurement is possible is because this approach largely does not rely on subject-matter expertise. When Holden and I started GiveWell, neither of us had any experience in philanthropy, so we looked for charities that we could evaluate through data and evidence that we could analyze, to make recommendations that we could fully explain. This led us to focus on organizations that had impacts that were relatively easy to measure.

The output of this process is reflected in our current top charities and the programs they run, which are analyzed in our intervention reports.

GiveWell has now been doing research to find the best giving opportunities in global health and development for 11 years, and we plan to increase the scope of giving opportunities we consider. We plan to expand our research team and scope in order to determine whether there are giving opportunities in global health and development that are more cost-effective than those we have identified to date.

We expect this expansion of our work to take us in a number of new directions, some of which we have begun to explore over the past few years. We have considered, in a few cases, the impact our top and standout charities have through providing technical assistance (for example, Deworm the World and Project Healthy Children), supported work to change government policies through our Incubation Grants program (for example, grants to the Centre for Pesticide Suicide Prevention and Innovation in Government Initiative), and begun to explore areas like tobacco policy and lead paint elimination.

Over the next several years, we plan to consider everything that we believe could be among the most cost-effective (broadly defined) giving opportunities in global health and development. This includes more comprehensively reviewing direct interventions in sectors where impacts are more difficult to measure, investigating opportunities to influence government policy, as well as other areas.

Making progress in areas where it is harder to determine causality will be challenging. In my opinion, we are excellent evaluators of empirical research, but we have yet to demonstrate the ability to make good judgments about giving opportunities when less empirical information is available. Our values, intellectual framework, culture, and the quality of our staff make me optimistic about our chances, but all of us at GiveWell recognize the difficulty of the project we are embarking on.

Our staff does not currently have the capacity or the capabilities to make enough progress in this direction, so we are planning to significantly increase the size of our staff. We have a research team of ten people, and we are planning to more than double in size over the next three years. We are planning to add some junior staff but are primarily aiming to hire people with relevant experience who can contribute as researchers and/or managers on our team.

GiveWell's top charities list is not going to change dramatically in the near future, and it may always include the charities we recommend today. Our top charities achieve outstanding, cost-effective results, and we believe they are some of the best giving opportunities in global health and development. We expect to conclude that many of the opportunities we consider in areas that are new for us are less cost-effective than those we currently recommend, but we also think it is possible that we will identify some opportunities that are much more cost-effective. We believe it is worth a major effort to find out.

Read More

The post How GiveWell’s research is evolving appeared first on The GiveWell Blog.

Elie

GiveWell’s money moved and web traffic in 2017

4 years ago

GiveWell is dedicated to finding outstanding giving opportunities and publishing the full details of our analysis. In addition to evaluations of other charities, we publish substantial evaluation of our own work. This post lays out highlights from our 2017 metrics report, which reviews what we know about how our research impacted donors. Please note:

  • We report on “metrics years” that run from February through January; for example, our 2017 data cover February 1, 2017 through January 31, 2018.
  • We differentiate between our traditional charity recommendations and the work of the Open Philanthropy Project, which became a separate organization in 2017 and whose work we exclude from this report.
  • More context on the relationships between GiveWell, Good Ventures, and the Open Philanthropy Project can be found here.

Summary of influence: In 2017, GiveWell influenced charitable giving in several ways. The following table summarizes our understanding of this influence.

Headline money moved: In 2017, we tracked $117.5 million in money moved to our recommended charities. Our money moved only includes donations that we are confident were influenced by our recommendations.

Money moved by charity: Our nine top charities received the majority of our money moved. Our seven standout charities received a total of $1.8 million.

Money moved by size of donor: In 2017, the number of donors and amount donated increased across each donor size category, with the notable exception of donations from donors giving $1,000,000 or more. In 2017, 90% of our money moved (excluding Good Ventures) came from 20% of our donors, who gave $1,000 or more.

Donor retention: The total number of donors who gave to our recommended charities or to GiveWell unrestricted increased about 29% year-over-year to 23,049 in 2017. This included 14,653 donors who gave for the first time. Among all donors who gave in the previous year, about 42% gave again in 2017, up from about 35% who gave again in 2016.

Our retention was stronger among donors who gave larger amounts or who first gave to our recommendations prior to 2015. Of larger donors (those who gave $10,000 or more in either of the last two years), about 73% who gave in 2016 gave again in 2017.

GiveWell’s expenses: GiveWell’s total operating expenses in 2017 were $4.6 million. Our expenses decreased from about $5.5 million in 2016 due to the Open Philanthropy Project becoming a separate organization in June 2017. We estimate that 67% of our total expenses ($3.1 million) supported our traditional top charity work and about 33% supported the Open Philanthropy Project. In 2016, we estimated that expenses for our traditional top charity work were about $2.0 million.

Donations supporting GiveWell’s operations: GiveWell raised $5.7 million in unrestricted funding (which we use to support our operations) in 2017, compared to $5.6 million in 2016. Our major institutional supporters and the six largest individual donors contributed about 49% of GiveWell’s operational funding in 2017.

Web traffic: The number of unique visitors to our website remained flat in 2017 compared to 2016 (when excluding visitors driven by AdWords, Google’s online advertising product).

For more detail, see our full metrics report (PDF).

The post GiveWell’s money moved and web traffic in 2017 appeared first on The GiveWell Blog.

Maryana Pinchuk

GiveWell’s outreach and operations: 2017 review and 2018 plans

4 years 2 months ago

This is the third of three posts that form our annual review and plan for the following year. The first two posts covered GiveWell’s progress and plans on research. This post reviews and evaluates GiveWell’s progress last year in outreach and operations and sketches out some high-level goals for the current year. A separate post will look at metrics on our influence on donations in 2017. We aim to release our metrics on our influence on donations in 2017 by the end of June 2018.

Summary

Outreach: Before 2017, outreach wasn’t a major organizational priority at GiveWell (more in this 2014 blog post). In our plans for 2017, we wrote that we planned to put more emphasis on outreach, but were at the early stages of thinking through what that might involve. In the second half of 2017, we experimented with a number of different approaches to outreach (more on the results below). In 2018, we plan to increase the resources we devote to outreach primarily by hiring a Head of Growth and adding staff to improve our post-donation follow-up with donors.

Operations: In 2017, we completed the separation of GiveWell and the Open Philanthropy Project and increased our operations capacity with three new hires. In 2018, our top priorities are to hire a new Director of Operations (which we have now done), maintain our critical functions, and prepare our systems for increased growth in outreach.

Outreach 2017 review and 2018 plans

Before 2017, outreach wasn’t a major organizational priority at GiveWell (more in this 2014 blog post). In our plans for 2017, we wrote that we planned to put more emphasis on outreach, but were at the early stages of thinking through what that might involve.

We currently have one staff member, Catherine Hollander, who works on outreach full-time. Two others, Tracy Williams and Isabel Arjmand, each spend significant time on outreach. From August 2017, our Executive Director, Elie Hassenfeld, also started to allocate a significant amount of his time to outreach.

How did we do in 2017?

In 2017, we focused on experimentation. In brief, we found that:

  • Advertising on podcasts has had strong results. Using the methodology described in this blog post, our best guess is that each dollar we spent on podcast advertising returned $5-14 in donations to our top charities.
  • Increasing the consistency of our communication with members of the media had strong results for the time invested.
  • Retaining a digital marketing consultant yielded strong results.
  • Retaining a PR firm to generate media mentions did not have positive results.
  • We’ve had a limited number of conversations with high net worth donors. We don’t yet have enough information to conclude whether this was a good use of time.

You can see our estimates of the five-year net present value of donations generated by each of these activities here. Overall, we spent approximately $200,000 and devoted significant staff time to this work. Our best estimate is that these efforts resulted in $2.5 million to $5.9 million in additional donations to our recommended charities.

We conclude:

  • New work on outreach had a high return on investment in 2017.
  • Some activities, such as podcast advertising and digital marketing improvements, have shown particularly strong results and should be scaled up.

What are our priorities for 2018?

Our marketing funnel has three stages:

  1. Awareness/acquisition: more people hear about GiveWell and visit the website,
  2. Conversion: more people who visit the site donate, and
  3. Retention: over time, donors maintain or increase their donations.

Our current working theory is that we should prioritize (though not exclusively) improving the bottom of this funnel (retention and conversion) before moving more people through it. We also plan to scale up the activities that worked well in 2017 and to continue experimenting with different approaches.

Our primary outreach priorities (which we expect to achieve and devote substantial capacity to) for 2018 are:

  1. Hire a Head of Growth to improve our efforts to acquire and convert new donors via our website. Over the long term, the Head of Growth will be responsible for digital marketing.

    What does success look like? Hire a Head of Growth.

  2. Improve the post-donation experience. We believe we have substantial room to improve our post-donation communication with donors. We have hired a consultant to help us improve our process.

    What does success look like? Significantly improve our process for post-donation follow-up before giving season 2018.

    At this point, we’re still in the earliest stages of figuring out how we’ll do this, so we don’t have concrete goals for the year beyond finalizing our plan in the next few months. Our stretch goal for the year is to succeed in achieving measured improvement in our dollar retention rate/lifetime value of each donor.

Our secondary outreach priorities (which we expect to achieve, but not devote substantial capacity to) for 2018 are:

  1. Continue advertising on podcasts. This advertising was particularly successful in 2017. We want to systematically assess podcast advertising opportunities and increase our podcast advertising. We plan to spend approximately $250,000 to $350,000 on podcast advertising this year.

    What does success look like? Advertise on new podcasts and measure results to decide how much to spend in 2019.

  2. Receive coverage in major news outlets. This has led to increased donations in the past.

    What does success look like? Pitch major news outlets on at least five stories in total and get at least one story covered.

  3. Deepen relationships with the effective altruism community. We want to deepen our relationships with groups in the effective altruism community doing outreach, particularly to high net worth donors.

For a list of other potentially promising projects we’re unlikely to prioritize this year, see this spreadsheet.

Operations 2017 review and 2018 plans

In 2017, we increased our operations staff capacity, made a number of changes to our internal systems, and completed the separation of GiveWell and the Open Philanthropy Project. In addition to maintaining critical functions, our highest priorities for 2018 are to (i) appoint a new Director of Operations and (ii) make improvements to our processes across the board to prepare our systems for major growth in outreach.

How did we do in 2017?

We made a number of improvements to our operations. In brief:

  • We completed the separation of GiveWell and the Open Philanthropy Project.
  • Donations: We hired two new members of our donations team, which allowed us to process donations consistently notwithstanding increased volume. We also added Betterment and Bitpay (for Bitcoin) as donation options.
  • Finance: We hired a Controller. We rolled out a few systems to improve the efficiency of our internal processes (Expensify, Bill.com, and others).
  • Social cohesion: We created a regular schedule for visit days for remote staff and staff events to maintain cohesion.

In January 2018, Sarah Ward, our former Director of Operations, departed. Natalie Crispin (Senior Research Analyst) has been covering her previous responsibilities during our search for a new hire to take them on.

What are our priorities for 2018?

In the first half of 2018, we aim to move from a situation in which we were maintaining critical functions to positioning the organization to grow.

Our two main priorities for the first half of 2018 are to:

  1. Appoint a new Director of Operations (complete). In April 2018, we hired Whitney Shinkle as our new Director of Operations. Between January and April 2018, Natalie Crispin served as our interim Director of Operations.
  2. Prepare our systems for major growth in outreach, which we expect to lead to increases in spending, staff, and donations.
  3. Maintain critical operations across domains: donations, finance, HR, office, website, recruiting, and staff cohesion.

Major operations projects we aim to complete in the first half of 2018 include:

  • A significant improvement in our approach to budgeting making it significantly easier for us to share updated actual spending versus budget.
  • We retained a compensation consultant to help us benchmark GiveWell staff compensation to comparable organizations.
  • We published our 2016 metrics report and plan to publish our 2017 money moved report by the end of June.

The post GiveWell’s outreach and operations: 2017 review and 2018 plans appeared first on The GiveWell Blog.

James Snowden (GiveWell)

Review of our research in 2017

4 years 2 months ago

This is the first of three posts that form our annual review and plan for the following year. This post reviews and evaluates last year’s progress on our work of finding and recommending evidence-based, thoroughly-vetted charities that serve the global poor. The following two posts will cover (i) our plans for GiveWell’s research in 2018 and (ii) GiveWell’s progress and plans as an organization. We aim to release our metrics on our influence on donations in 2017 by the end of June 2018.

Summary

We believe that 2017 was a successful year for GiveWell’s research. We met our five primary goals for the year, as articulated in our plan post from the beginning of the year:

Our primary research goals for 2017 are to:

  1. Speed up our output of new intervention assessments, by hiring a Senior Fellow and by improving our process for reviewing interventions at a shallow level.
  2. Increase the number of promising charities that apply for our recommendation. Alternatively, we may learn why we have relatively few strong applicants and decide whether to change our process as a result. Research Analyst Chelsea Tabart will spend most of her time on this project.
  3. Through GiveWell Incubation Grants, fund projects that may lead to more top charity contenders in the future and consider grantees No Lean Season and Zusha! as potential 2017 top charities.
  4. Further improve the robustness and usability of our cost-effectiveness model.
  5. Improve our process for following the progress of current top charities to reduce staff time, while maintaining quality. We also have some specific goals (discussed below) with respect to answering open questions about current top charities.

We achieved our five primary goals for the year:

  1. Our intervention-related output was greater than in any past year, although we still see room for improvement in the pace with which we complete and publish this work (more). We hired a Senior Fellow and published nine full or interim intervention reports in 2017, compared to four in 2016.
  2. We increased the number of promising charities that applied for our recommendation (more).
  3. We added two new top charities: Evidence Action’s No Lean Season (the first top charity to start as a GiveWell Incubation Grant recipient) and Helen Keller International’s vitamin A supplementation program (which joined our list as a result of our charity outreach work). We continued to follow our current Incubation Grant recipients and made several new Incubation Grants to grow the pipeline of new top charities (more).
  4. We made substantial improvements to our cost-effectiveness analysis (more).
  5. We reduced the amount of staff time spent on following our current top charities. We also completed 17 of the 19 activities outlined in last year’s plan (more).

We discuss progress on each of our primary goals below. For each high-level goal, we include (i) the subgoals we set in our last annual review, (ii) an evaluation of whether we met those subgoals, and (iii) a summary of key activities completed last year.

Goal 1: Speed up intervention assessments

In early 2017, we wrote:

In recent years, we have completed few intervention reports, which has limited our ability to consider new potential top charities. We plan to increase the rate at which we form views on interventions this year by:

  • Hiring a Senior Fellow (or possibly more than one). We expect a Senior Fellow to have a Ph.D. in economics, public health, or statistics or equivalent experience and to focus on in-depth evidence reviews and cost-effectiveness assessments of interventions that appear promising after a shallower investigation. In addition, Open Philanthropy Project Senior Advisor David Roodman may spend some more time on intervention related work.
  • Doing low-intensity research on a large number of promising interventions. We generally start with a two to four hour “quick intervention assessment,” and then prioritize interventions for a 20-30 hour “interim intervention report” (example). We don’t yet have a good sense of how many of these of these we will complete this year, because we’re unsure both about how much capacity we will have for this work and about how many promising interventions there will be at each step in the process.
  • Continuing to improve our systems for ensuring that we become aware of promising interventions and new relevant research as it becomes available. We expect to learn about additional interventions by tracking new research, particularly randomized controlled trials, in global health and development and by talking to select organizations about programs they run that they think we should look into.

How did we do? Achieved our goal.

Due to our uncertainty about the capacity we could devote to intervention assessments, we did not have an explicit target for how many reports we expected to complete. In 2017, we published seven interim intervention reports, two full intervention reports, and completed ~30 quick evidence assessments (defined below). Our research output for 2017 was higher than 2016, when we published one full intervention report, three interim intervention reports, and completed 30 quick evidence assessments.

What did we do?

Goal 2: Increase the pipeline of promising charities applying for our recommendation

In early 2017, we wrote:

We would like to better understand whether we have failed to get the word out about the potential value we offer or communicate well about our process and charities’ likelihood of success, or, alternatively, whether charities are making well-informed decisions about their fit with our criteria. (More on why we think more charities should consider applying for a GiveWell recommendation in this post.)

This year, we have designated GiveWell Research Analyst Chelsea Tabart as charity liaison. Her role is to increase and improve our pipeline of top charity contenders by answering charities’ questions about our process and which program(s) they should apply with, encouraging promising organizations to apply, and, through these conversations, understanding what the barriers are to more charities applying.

We aim by the end of the year to have a stronger pipeline of charities applying, have confidence that we are not missing strong contenders, or understand how we should adjust our process in the future.

How did we do? Achieved our goal.

More charities entered our top charity review process in 2017, although it’s unclear whether this was due to our charity liaison activities. Five charities formally applied in 2017, compared to two in 2016, and four in 2015. One of those charities, Helen Keller International’s vitamin A supplementation program, became a top charity.

While we feel our relationships with well-regarded global health and development implementers and funders have improved, we continue to feel limited in our ability to understand whether there are funding gaps for evidence-backed, highly cost-effective work within large international NGOs and multilateral aid organizations such as the Global Fund to Fight AIDS, Tuberculosis and Malaria.

What did we do?

  • We had at least one conversation with 42 organizations to introduce them to GiveWell’s work in 2017, compared to 16 in 2016.
  • Where organizations running multiple programs expressed interest in applying for our recommendation, we had several calls with them to help determine whether they should apply and which of their programs would be the most promising fit for a top charity evaluation. We had not offered this proactive support to organizations in the past.
  • We hosted two charity-focused events: (i) a conference call for charities with GiveWell senior staff to present an update on our work as it relates to charities and to give them a chance to ask questions directly of our senior team and (ii) a networking event for our recommended organizations in London.
  • We attended seven conferences on global health and development issues to broaden our network and perspective in subject-matter areas that GiveWell has not historically worked on.
Goal 3: Maintain Incubation Grants

In early 2017, we wrote:

We made significant progress on Incubation Grants in 2016 and plan in 2017 to largely continue with ongoing engagements, while being open to new grantmaking opportunities that are brought to our attention.

Among early-to-mid stage grants, we plan to spend the most time on working with IDinsight and New Incentives (where our feedback is needed to move the projects forward), and a smaller amount of time on Results for Development and Charity Science: Health (where we are only following along with ongoing projects).

Another major priority will be following up on two later-stage grantees, No Lean Season and Zusha!, groups that are contenders for a top charity recommendation in 2017. For No Lean Season, a program run by Evidence Action, our main outstanding questions are whether the program will have room for more funding in 2018 and whether monitoring will be high quality as the program scales. We have similar questions about Zusha! and in addition are awaiting randomized controlled trial results that are expected later this year.

How did we do? Exceeded goal.

As expected, our work last year focused on following up on current grantees. No Lean Season, one of our later-stage grantees, graduated to top charity status and we made one grant to a new grantee, the Centre for Pesticide Suicide Prevention. We also made a number of grants to improve our understanding of the evidence base for our priority programs and deepened our partnership with IDinsight.

What did we do?

Goal 4: Improve our cost-effectiveness analysis

In early 2017, we wrote:

We plan to continue making improvements to our cost-effectiveness model and the data it draws on (separate from adding new interventions to the model, which is part of the intervention report work discussed above). Projects we are currently prioritizing include:

  • Making it more straightforward to see how personal values are incorporated into the model and what the implications of those values are.
  • Revisiting the prevalence and intensity adjustment that we use to compare the average per-person impact of deworming in places that our top charities work to the locations where the studies that found long-term impact of deworming were conducted. More in this post.
  • Improving the insecticide-treated nets model by revisiting how it incorporates effects on adult mortality and adjustments for regions with different malaria burdens and changes in malaria burden over time.

How did we do? Achieved goal.

We made substantial progress on improving our cost-effectiveness analysis in 2017.

What did we do?

  • Moved to a system of making more frequent updates to our cost-effectiveness analysis. This has made it easier to identify which specific factors are driving changes in the estimated cost-effectiveness of our top charities.
  • Revisited how we think about leverage and funging (how donating to our top charities influences how other funders spend their money) and updated our cost-effectiveness analysis accordingly.
  • Published a report on how other global actors approach the difficult moral tradeoffs we face.
  • Prior to announcing our 2017 recommendations, we performed a sensitivity check on our cost-effectiveness analysis to identify how sensitive our final outputs were to different uncertain inputs. This has helped us identify which inputs we should prioritize additional research on, and we believe it has made our communication more transparent, particularly around our personal values.
  • Revisited and updated our prevalence and intensity adjustments for deworming.
  • Deprioritized improving how our insecticide-treated net model incorporates effects on adult mortality. A limited number of conversations with malaria experts made us less confident that there was informative research on the question that would improve the accuracy of our models.
  • Deprioritized making adjustments for subnational regions with different malaria burdens because it would take substantial time to deeply understand the assumptions informing the subnational models we have seen. We believe this remains an important weakness of our model and that it limits our ability to make high-quality decisions about prioritization among different regional funding gaps.
Goal 5: Improve our process for following top charities

In early 2017, we wrote:

“In 2017, we plan to have a single staff member do most of this work and expect it to take a half to two-thirds of a full-time job. Three other staff will spend a small portion of their time, totaling approximately the equivalent of one full-time job, on this work.”

How did we do? Achieved goal.

We estimate that it took about 40 percent of the staff member’s time who focused on this work plus a small portion of four other staff members’ time, totaling at most and likely somewhat less than the equivalent of a full-time job (roughly half the time we dedicated to top charity updates in 2016).

We believe we maintained or increased the quality of the top charity updates, as we completed or made major progress on all but two of the activities and questions outlined in last year’s plan.

What did we do?

The table below summarizes our progress on each of the activities and open questions outlined in last year’s plan.

Charity Goals and open questions from 2017 plan Did we meet our goal? What did we do? Evidence Action’s Deworm the World Initiative “We have now followed these groups for several years and do not have major outstanding questions about them. We plan to ask for updates on financial information, monitoring results, and room for more funding and have regular phone calls with them to learn about operational changes that might lead us to ask additional questions.” Yes We had regular phone calls, received up-to-date financial information, updated room for more funding, and reviewed new monitoring information from Nigeria, Vietnam, Kenya, and Ethiopia (see rows 11-20 and an overview of what we learned). GiveDirectly “We have now followed these groups for several years and do not have major outstanding questions about them. We plan to ask for updates on financial information, monitoring results, and room for more funding and have regular phone calls with them to learn about operational changes that might lead us to ask additional questions.” Yes We had regular phone calls, received up-to-date financial information, updated room for more funding, and reviewed new monitoring information from Kenya (1, 2). (Overview of what we learned.) Schistosomiasis Control Initiative “We have now followed these groups for several years and do not have major outstanding questions about them. We plan to ask for updates on financial information, monitoring results, and room for more funding and have regular phone calls with them to learn about operational changes that might lead us to ask additional questions.” Yes We had regular phone calls, received up-to-date financial information, updated room for more funding, and reviewed new monitoring information from 2016 programs in a number of countries. (Monitoring information, overview of what we learned.) Against Malaria Foundation (AMF) “Will AMF’s monitoring processes be high quality?” Yes We commissioned IDinsight, an organization with which we are partnering as part of our Incubation Grants program, to observe post-distribution surveys in Malawi and Ghana and report their findings. “Going forward, AMF aims to fund larger distributions and commit funding further ahead of when a distribution is scheduled to occur than it has, for the most part, done in the past. Will this increase the extent to which AMF funds displace funds from other sources, or will there continue to be evidence that AMF’s funds are largely adding to the total number of nets distributed?” Partial We learned relatively little about the displacement/fungibility question because AMF signed relatively few new agreements to fund long-lasting insecticide-treated net distributions in 2017. There was an update to how AMF will be tracking displacement, described in the second paragraph here. “In order to estimate AMF’s room for more funding, we will seek out information on the location and size of funding gaps for mass net distribution campaigns from AMF, the African Leaders Malaria Alliance, and possibly other funders of nets. As we have in the past, we will use this information in conjunction with conversations with AMF about non-funding bottlenecks to its ability to fill various gaps.” Yes We got updates on AMF’s room for more funding, as summarized in this post. The END Fund’s deworming program “We have not yet seen monitoring on par with that from our other top charities from the END Fund. We expect results from coverage surveys from END Fund programs this year. Will these surveys be high quality and demonstrate that the END Fund is funding successful programs?” Yes We saw some monitoring from END Fund programs; previously our recommendation of the END Fund was based on specific monitoring plans that we found credible (more here). “We have not yet tried to compare the cost-effectiveness of the END Fund to our other top charities in our cost-effectiveness model. We will be seeking additional information from the END Fund about cost per treatment and baseline infection rates” Yes We significantly improved our understanding of the END Fund’s cost per treatment and the baseline prevalence in areas where the END Fund works. We completed a cost-effectiveness analysis, though we continue to have lower confidence in our estimates than we do for the deworming organizations that we have recommended for several years. “Questions around room for more funding: the extent to which funding due to GiveWell’s recommendation increases the amount that the END Fund spends on deworming versus other programs, actual and projected revenue from other sources, and what deworming grantmaking opportunities the END Fund expects to have.” Yes We estimated the extent to which funding due to GiveWell’s recommendation increases the amount that the END Fund spends on deworming versus other programs, discussed here. “We visited the END Fund’s programs in Rwanda and Idjwi island, DRC in January 2017 and will publish notes and photos from our visit shortly.” Yes We posted notes and photos from our site visit here. Malaria Consortium’s seasonal malaria chemoprevention program “Further research on the evidence of effectiveness, cost-effectiveness, and potential downsides of seasonal malaria chemoprevention (SMC) (due to time constraints we have not yet completed a full intervention report, though we felt sufficiently confident in the intervention to recommend Malaria Consortium).” Yes We reviewed each of the RCTs included in the Cochrane review for seasonal malaria chemoprevention, and possible negative/offsetting impacts. We updated our interim intervention report to a full intervention report and added new information to our cost-effectiveness analysis. Our key conclusions did not change substantially and SMC remains a priority program. “Getting a better understanding of the methodology Malaria Consortium uses for estimating coverage rates.” Yes We spoke with Malaria Consortium to understand how they measure coverage and updated our cost-effectiveness analysis to account for different levels of coverage in the Malaria Consortium program relative to the headline results of the RCTs in the Cochrane review (conversation notes here). “Completing a more in-depth room for more funding analysis for the program for 2018 than we did for 2017.” Yes We completed a significantly more in-depth room for more funding analysis than we had previously (more here). “We may visit a Malaria Consortium seasonal malaria chemoprevention program in summer 2017.” No We did not conduct a site visit. Sightsavers’ deworming program “We expect to make limited progress this year because the first deworming mass drug administration funded with GiveWell-influenced funds is not expected to take place until September at the earliest and monitoring results aren’t expected until early 2018. Because Sightsavers has done fairly little deworming in the past year, we don’t expect to be able to learn much from its ongoing programs.” Exceeded In 2017, as expected, we learned relatively little about the performance of Sightsavers’ deworming programs, because programs funded with GiveWell-directed funds were at early stages. We did not expect to receive any monitoring results from programs funded with GiveWell-directed funds; however, Sightsavers shared a coverage survey from Guinea with us earlier than expected. The survey found middling coverage results. “Getting more information from Sightsavers about baseline prevalence and intensity of worm infections in the areas it is working, to inform our cost-effectiveness analysis.” Yes We significantly improved our understanding of Sightsavers’ cost per treatment and the baseline prevalence in areas where Sightsavers works (which is used in our cost-effectiveness analysis). “Using Sightsavers’ budget for the projects and planned treatment numbers to improve our estimate of the cost per treatment – another input into our cost-effectiveness analysis.” Yes We significantly improved our understanding of Sightsavers’ cost per treatment and the baseline prevalence in areas where Sightsavers works (which is used in our cost-effectiveness analysis). “Completing a room for more funding analysis for 2018.” Yes We completed a room for more funding analysis (more here). Standout charities “We plan to have at least one phone call with each of these groups to discuss whether anything has changed that might lead us to reopen consideration of the organization as a potential top charity” Yes We spoke with each standout charity. Conversation notes here: Development Media International, Food Fortification Initiative, Global Alliance for Improved Nutrition’s Universal Salt Iodizational program, Iodine Global Network, Living Goods, and Project Healthy Children.

The post Review of our research in 2017 appeared first on The GiveWell Blog.

James Snowden (GiveWell)

GiveWell’s money moved and web traffic in 2016

4 years 3 months ago

In September 2017, we posted an interim update on GiveWell’s 2016 money moved and web traffic. This post summarizes the key takeaways from our full 2016 money moved and web traffic metrics report. Note that some of the numbers, including the total headline money moved, have changed since our interim report. Since then, we decided to exclude some donations from our headline money moved figure (details in the full report), and we corrected some minor errors.

This report was highly delayed (as discussed in the interim update). We expect to publish our report on GiveWell’s 2017 money moved and web traffic much more quickly; our current expectation is that we will publish that report by the end of June.

GiveWell is dedicated to finding outstanding giving opportunities and publishing the full details of our analysis. In addition to evaluations of other charities, we publish substantial evaluation of our own work. This post lays out highlights from our 2016 metrics report, which reviews what we know about how our research impacted donors. Please note:

  • We report on “metrics years” that run from February through January; for example, our 2016 data cover February 1, 2016 through January 31, 2017.
  • We differentiate between our traditional charity recommendations and our work on the Open Philanthropy Project, which became a separate organization in 2017 and whose work we exclude from this report.
  • More context on the relationship between Good Ventures and GiveWell can be found here.

Summary of influence: In 2016, GiveWell influenced charitable giving in several ways. The following table summarizes our understanding of this influence.

Headline money moved: In 2016, we tracked $88.6 million in money moved to our recommended charities. Our money moved only includes donations that we are confident were influenced by our recommendations.

Money moved by charity: Our seven top charities received the majority of our money moved. Our six standout charities received a total of $2.9 million.

Money moved by size of donor: In 2016, the number of donors and amount donated increased across each donor size category, with the notable exception of donations from donors giving $1,000,000 or more. In 2016, 93% of our money moved (excluding Good Ventures) came from 19% of our donors, who gave $1,000 or more.

Donor retention: The total number of donors who gave to our recommended charities or to GiveWell unrestricted increased about 16% year-over-year to 17,834 in 2016. This included 12,461 donors who gave for the first time. Among all donors who gave in the previous year, about 35% gave again in 2016, down from about 40% who gave again in 2015.

Our retention was stronger among donors who gave larger amounts or who first gave to our recommendations prior to 2014. Of larger donors (those who gave $10,000 or more in either of the last two years), about 77% who gave in 2015 gave again in 2016.

GiveWell’s expenses: GiveWell’s total operating expenses in 2016 were $5.5 million. Our expenses increased from about $3.4 million in 2015 as the size of our staff grew and average seniority level rose. We estimate that about one-third of our total expenses ($2.0 million) supported our traditional top charity work and about two-thirds supported the Open Philanthropy Project. In 2015, we estimated that expenses for our traditional charity work were about $1.1 million.

Donations supporting GiveWell’s operations: GiveWell raised $5.6 million in unrestricted funding (which we use to support our operations) in 2016, compared to $5.1 million in 2015. Our major institutional supporters and the five largest individual donors contributed about 70% of GiveWell’s operational funding in 2016. This is driven in large part by the fact that Good Ventures funded two-thirds of the costs of the Open Philanthropy project, in addition to funding 20% of GiveWell’s other costs.

Web traffic: The number of unique visitors to our website was down very slightly (by 1%) in 2016 compared to 2015 (when excluding visitors driven by AdWords, Google’s online advertising product).

For more detail, see our full metrics report (PDF).

The post GiveWell’s money moved and web traffic in 2016 appeared first on The GiveWell Blog.

Natalie Crispin

Update on our work on outreach

4 years 6 months ago

GiveWell’s impact is a function of the quality of our research and the amount of money we direct to our recommended charities (our “money moved”). Historically, we’ve focused mostly on research because we felt that the quality of our recommendations was a greater constraint to our impact than our money moved.

This has changed. Outreach is now a major organizational priority. The goal of this work is to increase the amount of money we direct to our top-recommended charities.

In April 2014 I wrote about our work on outreach to explain why we hadn’t prioritized it: in brief, our growth had largely been driven by inbound interest in GiveWell, and proactive outreach efforts (beyond building relationships with existing donors) hadn’t yielded results that were worth the cost.

What changed?

  • We believe that the amount of money we move is now a greater constraint to our impact than additional improvements in the quality of our research. Over the last two years, we’ve added five new top charities (three of which implement programs that weren’t previously represented on our top charities list), and we expect that our top charities, collectively, will have more than $200 million in unfilled funding gaps once they’ve received the funding that we expect to direct to them. (This calculation excludes GiveDirectly, which we believe could absorb and distribute hundreds of millions of dollars.) At the same time, the quality of our research and our capacity for research is higher than it’s ever been, so the returns to adding staff there (in terms of the pace at which we identify significantly better giving opportunities) are now lower.
  • Increased capacity for outreach. In our 2014 post, we wrote that one of our key constraints was that senior staff (which at the time meant primarily GiveWell Co-Founder Holden Karnofsky and me) were necessary for most outreach-related work. This has changed. We now have capacity to take on outreach work as other staff have been hired and trained on this type of work.
  • Better information on the impact of GiveWell’s outreach. We now have better information about the returns to outreach because:
    1. We’ve collected better data (via an improved donations processing system and outreach efforts) about where donors find out about us. Because of our ability to track donors, we know that a single appearance on NPR or major podcasts tends to drive $50,000+ in annual donations.
    2. More time passing has demonstrated that the lifetime value of the donations of a first time donor is higher than we expected. In several cases, we’ve seen major donors (i.e., those giving $10,000-$100,000) increase their annual giving by a factor of 10 or more.

We’re in the early stages of figuring out how we can proactively invest time and money in outreach to significantly increase our money moved. For now, we’ve taken some opportunities that we think will have positive returns; these are the three that we’ve invested the most time and money in to date:

  • Podcast advertising. We’ve been advertising on podcasts that we believe our target audience listens to, based on interviews with current donors and GiveWell staff. In February and March, we ran a small experiment with a few ads on FiveThirtyEight’s Politics podcast and Vox’s The Weeds.1We’ve also been running ads on Julia Galef’s Rationally Speaking podcast since then. Because it’s much smaller and more targeted, we’ve excluded it from this analysis. Measured returns to advertising on Rationally Speaking have been significantly better than the more mainstream podcasts discussed in this post. jQuery("#footnote_plugin_tooltip_1").tooltip({ tip: "#footnote_plugin_tooltip_text_1", tipClass: "footnote_tooltip", effect: "fade", fadeOutSpeed: 100, predelay: 400, position: "top right", relative: true, offset: [10, 10] });

    In total, we spent approximately $20,000 on ads for this initial experiment. We ask donors who give via our website to tell us where they learned about GiveWell when they donate. GiveWell received approximately $8,000 in donations between February 1 and November 20 from donors who reported that they had learned about us via these podcasts.

    The donations we received were from first-time donors; to assess the impact of our advertising, we need to estimate the lifetime value of acquiring a new donor. In work we’ve done to assess our retention rate, we’ve seen that (a) approximately 20-25% of the donors who make a first-time donation of less than $1,000 give again in the subsequent year but (b) because many first-time donors increase the size of their donation over time, collectively, the donors who recur give more than 100% of the value of what they give in their first year.

    At higher donation levels ($1,000-$100,000), we measure 40-45% retention among donors, which leads to retention of approximately two-thirds of dollars given.2I say “measure” retention because we’ve learned that many donors give subsequent donations directly to our top charities and don’t report those donations to us. We’ve tried to follow up with lapsed donors and with charities to track these donors down. jQuery("#footnote_plugin_tooltip_2").tooltip({ tip: "#footnote_plugin_tooltip_text_2", tipClass: "footnote_tooltip", effect: "fade", fadeOutSpeed: 100, predelay: 400, position: "top right", relative: true, offset: [10, 10] });

    We therefore estimated the net present value of expected future donations (over the next five years) from these podcasts ads as somewhere between approximately $20,000 (assuming two-thirds dollar retention for the first two years and 100% dollar retention subsequently) and $45,000 (assuming 100% dollar retention).3We only projected donations over five years. This is fairly arbitrary because we don’t have long-term enough data to know whether or not this is a reasonable assumption. We capped it to prevent our assessment being driven by speculation about how much money would be donated many years in the future. jQuery("#footnote_plugin_tooltip_3").tooltip({ tip: "#footnote_plugin_tooltip_text_3", tipClass: "footnote_tooltip", effect: "fade", fadeOutSpeed: 100, predelay: 400, position: "top right", relative: true, offset: [10, 10] });

    A few additional facts are worth keeping in mind about the above figures:

    • We ran this experiment in February and March; most donors give at the end of the calendar year. We consistently see donors who find out about GiveWell during the course of the year, but donate in December. Other things equal, we expect that our advertising would have had greater measured returns in December than earlier in the year.
    • We are only able to track donors who (a) fill out our donation form telling us where they learned about us and (b) give directly through our website rather than to our top charities. Less than 50% of donors who give via credit card (and a smaller percentage of donors who give via check) tell us where they learned about GiveWell. Also, roughly speaking, approximately 50% of the donors and dollars we influence come through GiveWell rather than going to our top charities.4I took this rough estimate from footnote 26, on page 15, of GiveWell’s 2015 metrics report. jQuery("#footnote_plugin_tooltip_4").tooltip({ tip: "#footnote_plugin_tooltip_text_4", tipClass: "footnote_tooltip", effect: "fade", fadeOutSpeed: 100, predelay: 400, position: "top right", relative: true, offset: [10, 10] });
    • It’s certainly possible that donors who learn about us via podcast would be more likely to give through our website than an average donor, more likely to report on how they found us (since their source is clear), or less likely to be retained. My best guess is that donors who learn about us via podcast ads behave similarly to our other donors, but I won’t be surprised if they don’t.

    With all that in mind, I believe that the impact of our podcast advertising is higher than what we directly measured.

    The results we saw from February to November this year were promising enough that we decided to increase the size of our experiment by spending approximately $100,000 on podcast ads. We’re currently running ads on FiveThirtyEight’s Politics podcast and Ezra Klein’s podcast and The Weeds at Vox.

  • Earned media outreach. Mentions of GiveWell in the media have historically been a strong driver of growth. We aimed to increase mentions of GiveWell in high-quality, high-profile media where we’ve had the most past success as measured by dollars donated (i.e., media like The New York Times, NPR, The Wall Street Journal, and Financial Times). We retained a PR firm that came strongly recommended; we also increased 1-to-1 outreach by GiveWell staff to members of the media who have covered GiveWell in the past. It’s very hard to attribute the impact of the additional effort we’ve invested—overall, our effort has been fairly limited, and it’s hard to easily draw the causal lines between our work and the stories that appear—but my guess is that our increased efforts have led to more coverage of GiveWell and our top charities this giving season than in the recent past.
  • Website improvements. Companies that sell products online invest significant effort into optimizing their websites and checkout pages to maximize their revenues. We retained a marketing consultant, Will Wong of Mission Street, and we’ve been A/B testing different donation pages and plan to test other pages on our website such as our homepage or top charities page to see whether we can increase our conversion rate (i.e., the percentage of visitors to our website who give to one of our top charities). For context, our current conversion rate is 1%. Our understanding is that a standard conversion rate for e-commerce companies is 2%, and that international nonprofits have a similar conversion rate.5See Pg 51 of the study downloadable here. jQuery("#footnote_plugin_tooltip_5").tooltip({ tip: "#footnote_plugin_tooltip_text_5", tipClass: "footnote_tooltip", effect: "fade", fadeOutSpeed: 100, predelay: 400, position: "top right", relative: true, offset: [10, 10] }); An increase in our conversion rate to the industry average would lead to a significant increase in the amount of money we direct to our top charities.

Notes   [ + ]

1. ↑ We’ve also been running ads on Julia Galef’s Rationally Speaking podcast since then. Because it’s much smaller and more targeted, we’ve excluded it from this analysis. Measured returns to advertising on Rationally Speaking have been significantly better than the more mainstream podcasts discussed in this post. 2. ↑ I say “measure” retention because we’ve learned that many donors give subsequent donations directly to our top charities and don’t report those donations to us. We’ve tried to follow up with lapsed donors and with charities to track these donors down. 3. ↑ We only projected donations over five years. This is fairly arbitrary because we don’t have long-term enough data to know whether or not this is a reasonable assumption. We capped it to prevent our assessment being driven by speculation about how much money would be donated many years in the future. 4. ↑ I took this rough estimate from footnote 26, on page 15, of GiveWell’s 2015 metrics report. 5. ↑ See Pg 51 of the study downloadable here. function footnote_expand_reference_container() { jQuery("#footnote_references_container").show(); jQuery("#footnote_reference_container_collapse_button").text("-"); } function footnote_collapse_reference_container() { jQuery("#footnote_references_container").hide(); jQuery("#footnote_reference_container_collapse_button").text("+"); } function footnote_expand_collapse_reference_container() { if (jQuery("#footnote_references_container").is(":hidden")) { footnote_expand_reference_container(); } else { footnote_collapse_reference_container(); } } function footnote_moveToAnchor(p_str_TargetID) { footnote_expand_reference_container(); var l_obj_Target = jQuery("#" + p_str_TargetID); if(l_obj_Target.length) { jQuery('html, body').animate({ scrollTop: l_obj_Target.offset().top - window.innerHeight/2 }, 1000); } }

The post Update on our work on outreach appeared first on The GiveWell Blog.

Elie

Interim update on GiveWell’s money moved and web traffic in 2016

4 years 9 months ago

In 2016, we tracked a total of $91.6 million given to our top charities as a direct result of our research.

In addition to this $91.6 million, we also directed $13.3 million to our Incubation Grants program.

A note about this report:

We have yet to complete all of the work necessary to publish our 2016 metrics report. There are a number of reasons the report has been difficult to produce this year. In our view the main factors responsible for this delay were the increasing number and complexity of the data sources used to track donors giving to our recommended charities and competing priorities that required the attention of the staff member who produced the 2016 report. The delay was not a result of a decreased dedication to transparency.

In mid-September, we committed to publishing an update on our key metrics (money moved and web traffic) by October 1st even if we were unable to complete our report by then. We have failed to complete our full report, and, today, are publishing an interim update consistent with our commitment.

Unfortunately, we don’t have an updated estimate about when we’ll publish our full metrics report. It is possible that it will take us several months or more to complete it. We view this as a major failing on our part, and we plan to correct this in the future.

For the purpose of this report, please note:

  • We report on “metrics years” that run from February through January; for example, our 2016 data cover February 1, 2016 through January 31, 2017.
  • We differentiate between our traditional charity recommendations, the work of the Open Philanthropy Project, and our work aiming to support the development of future GiveWell top charities. GiveWell and Open Philanthropy are now separate legal organizations, but during 2016 Open Philanthropy was part of GiveWell, so we report its grantmaking here.
  • More context on the relationship between Good Ventures, Open Philanthropy, and GiveWell can be found here.

Summary of influence:

In 2016, GiveWell influenced charitable giving in several ways. The following table includes (a) donations from donors who cited our research when donating to a third party (or cited a source that recommended our top charities because of our recommendation), (b) donations to GiveWell that we granted to top charities and standout organizations, and (c) grants made on our recommendation, through GiveWell Incubation Grants and the Open Philanthropy Project.

Total money moved:

In 2016, GiveWell tracked $91.6 million in money moved to our recommended charities. Our money moved figure only includes donations that we are confident were influenced by our recommendations. In our full metrics report, we plan to include our best guess of the total funding that was given to our top and standout charities due to our research; the numbers in this blog post include only donations that we could specifically track as being due to our research. The methodology used to generate the numbers in this post was very similar to what is described in our 2015 metrics report, with the exception that, for 2016, we counted more donations through partner organizations such as Giving What We Can. We now believe that we should have included those donations last year.

Open Philanthropy gave an additional $13.3 million to the GiveWell Incubation Grants program, to support the development of future top charities, and GiveWell granted $400,000 in participation grants to organizations that applied for a top charity recommendation, from funding provided by Good Ventures for this purpose. We do not count either of these grant types in our headline money moved figure.

Money moved by charity:

Our seven top charities received the majority of our money moved. Our six standout charities received a total of about $3.1 million.

The post Interim update on GiveWell’s money moved and web traffic in 2016 appeared first on The GiveWell Blog.

Devin Jacob

Mid-year update on GiveWell’s progress

4 years 11 months ago

This post will provide a brief overview of GiveWell’s progress in a number of areas so far this year. In summary,

  1. Research: We are making progress on reaching charities that might be a good fit for a GiveWell recommendation and asking them to apply. We are also moving forward with GiveWell’s intervention prioritization goals.
  2. Operations: The separation of GiveWell and the Open Philanthropy Project was a major organizational priority in the first half of the year and was finalized on June 1. We’ve also increased the specialization on the operations team and outsourced some of GiveWell’s operations work.
  3. Outreach: Outreach is now a major organizational priority. We hope to develop a strategy for significantly increasing money moved to our recommended charities by September.

We hope you will let us know if you have any questions about our work this year in the comments.

Research

Our work on research now falls into two primary categories: Traditional top charities work, consisting of research into promising programs and evaluations of charities implementing them as potential GiveWell top charities, and GiveWell Incubation Grants, our work to grow the pipeline of potential top charities and improve our understanding of our current recommended charities.

Top charities

  • Encouraging charities to apply for a GiveWell recommendation. In recent years, we were surprised by how few charities reached out to GiveWell to apply for a recommendation. We guessed that some part of this may be driven by (a) a lack of understanding of GiveWell’s research priorities and which organizations we might be interested in recommending, and (b) a lack of understanding or misconceptions about GiveWell’s charity review process or the value added of a GiveWell recommendation in increasing a charity’s funding.

    We took two steps to address this problem in 2017. First, GiveWell Research Analyst Chelsea Tabart is now serving as GiveWell’s “charity liaison.” In this role, Chelsea connects with groups that may be a good fit for a GiveWell recommendation to learn more about their work and to encourage them to apply if a fit seems promising, and to explain GiveWell’s review process and value added. Second, we published a blog post on why we think more charities should consider applying for a GiveWell recommendation. We are now considering a number of charities as potential top-charity contenders and attribute this in part to taking the steps described above.

  • Intervention prioritization. A major goal this year is to assess a large number of interventions as potential GiveWell priority programs. This “intervention prioritization” work involves surveying the literature for a variety of interventions to identify the most cost-effective and evidence-backed programs.

    We have made good progress on intervention research in 2017. (We plan to write about our progress in more detail in a future post.) We completed 50 quick evidence assessments in the first half of the year and published interim intervention reports on:

    We also published an intervention report on surgery to repair obstetric fistula.

GiveWell Incubation Grants

We continue to expand our work on GiveWell’s Incubation Grants program:

Recruiting

We hired Caitlin McGugan as a Senior Fellow and James Snowden started working with us as a research consultant. We hope they will increase GiveWell’s output of intervention reports. We also have one summer research analyst, Scott Weathers, working with us.

Operations

The separation of GiveWell and the Open Philanthropy Project was a major priority for GiveWell’s operations team in the first half of 2017. The separation was finalized on June 1.

The operations team continues to increase in specialization (historically, GiveWell operations work has been done by generalists on staff). We hired Maryana Pinchuk to serve as a Donations Manager and Erin Wolff as a Donations Relations Assistant; a search for a Controller to manage our finance and accounting is underway. We have also started to work with new vendors to outsource some operations work, which we hope will increase our available staff capacity and improve the quality of our operations.

Outreach

Outreach is now a major organizational priority for GiveWell. In the past, we focused very little on efforts to reach new potential donors with GiveWell’s work. Now, we think outreach is more of a limiting factor than research—the high-value funding gaps we’ve identified exceed the amount of donations we expect to direct to those gaps.

We have developed a list of ideas for how to significantly increase the money GiveWell directs to our recommended charities, and are planning to work on the most promising ideas over the next few months. For example, we think there may be relatively low-intensity steps to take in areas like podcast advertising; in February, we advertised on a small number of podcasts and plan to do so again based on the cost of running advertisements and the additional donations to top charities we tracked as a direct result of those ads.

We are also trying to hire another Research Analyst, Outreach Focus to expand our capacity to communicate with donors and other individuals who rely on GiveWell’s research.

The post Mid-year update on GiveWell’s progress appeared first on The GiveWell Blog.

Catherine

GiveWell as an organization: progress in 2016 and plans for 2017

5 years 3 months ago

This is the third of four posts that form our annual review and plan for the following year. This post reviews and evaluates GiveWell’s progress last year as an organization and sketches out some high level goals for the current year. The first two posts covered GiveWell’s progress and plans on research. The last post in the series will look at metrics on our influence on donations in 2016.

First, a point of clarification. GiveWell as a legal entity currently employs both (a) staff whose work is described on givewell.org (finding outstanding evidence-backed, cost-effective programs) and (b) staff who work on the Open Philanthropy Project. We expect Open Philanthropy to become a separate organization this year (more below), pending board approval. The scope of this post is limited to (a) – the parts of the organization that will not become part of Open Philanthropy. Open Philanthropy has written about its progress and plans in this post.

Below, we first note three high-level points about where GiveWell is as an organization today. We then reflect on four questions that are important for thinking about our performance as an organization:

  • Do we have sufficient staff capacity?
  • Does our impact justify our operating expenses?
  • Does GiveWell have a positive and accurate public image?
  • Are we in a stable financial position?

Major organizational developments

Separation of the Open Philanthropy Project

We had aimed to complete the transition of Open Philanthropy staff to a new entity by the end of 2016 and did not accomplish this goal, though we are now effectively operating as two separate teams. We now expect, pending board approval, to complete the legal split by mid-2017. After the split, there will continue to be some shared staff between the organizations (GiveWell staff will track the time they spend on work for Open Philanthropy and GiveWell will bill Open Philanthropy for the time). We will continue to share office space.

GiveWell as an entity currently employs 35 staff members. After the split, we anticipate that GiveWell will continue to employ 15-20 of the current employees and that Elie Hassenfeld will remain as Executive Director of GiveWell. Holden Karnofsky, Co-Founder of GiveWell, currently spends very little time on GiveWell and will work full time for Open Philanthropy.

Outreach is now more of a limiting factor than research

We’ve gone from feeling that we had more funding available than we had good giving opportunities to a situation where we believe that strong giving opportunities have surpassed available funding. We estimate that we left over $100 million worth of very strong opportunities (top charity execution level 1 or 2 gaps, excluding GiveDirectly) unfilled last year.

This is due to increased research output (we added three new top charities and two new standouts) in 2016, an expectation of increased research output in the future (from our standard process and Incubation Grants), and decreased expectations of funding from Good Ventures. In a change from the previous year, Open Philanthropy’s tentative guess is currently that the “last dollar” it will give (from the pool of currently available capital) has higher expected value than gifts to GiveWell’s top charities today, leading it to recommend that Good Ventures cap its giving to GiveWell’s top charities at $50 million in 2016.

We expect to put more emphasis on expanding our outreach to potential donors interested in following our recommendations in 2017 than we have in past years. We are at early stages of thinking through what that might involve.

Organizational maturity

GiveWell will be 10 years old this year and we feel that we’ve reached a relatively stable place in our development. We are now making a major effort to strengthen our organizational infrastructure through filling specialized roles, particularly in operations (finance, donations management, technology, etc.); formalizing policies and procedures; and creating contingency plans for replacing senior staff.

Four key questions

Below we pose and respond to four questions about how we are doing as an organization.

Do we have sufficient staff capacity?

Operations: To date we have not had sufficient capacity for operations and have been slower to make improvements to our systems than we would have liked. In the last year, we have begun to make major changes to GiveWell’s operations team to try to correct for this. Sarah Ward was named Director of Operations, a new role, and we are pursuing a strategy of (a) hiring specialized firms to handle more of the HR and IT work that generalist staff have done in the past; (b) replacing our external accountants and auditors with firms that specialize in non-profits; and (c) moving current staff into and hiring for specialized roles, such as a donations manager, donor relations assistant, controller, and office manager. Our number of generalist operations staff has decreased; we expect to continue to have a need for a small number of generalist staff to manage relationships with external firms and fill gaps between specialist domains.

Our current operations team includes a Director of Operations, two operations generalists (who work on the website, accounting, recruiting, personnel management, donation processing, and IT), an Office Manager, an Administrative Assistant, a Donations Manager, a Donations Assistant, and a Donor Relations Assistant. We are hiring for an Operations and Legal Program Manager and expect to hire for additional roles in the coming months. After the expected spinoff of Open Philanthropy into a separate organization, the office manager, administrative assistant and one of the operations generalists will divide their time between the two organizations and Sarah will manage operations for both organizations temporarily; Open Philanthropy will begin building a separate operations team this year.

Research: Seven staff work on GiveWell’s research full time or close to full time. Elie Hassenfeld, GiveWell’s Executive Director, spends about half his time on GiveWell research. Elie spends the other half of his time on a combination of the Open Philanthropy project (about 20% of his time currently) and overseeing outreach, recruiting, and operations for GiveWell.

Josh Rosenberg and I have taken over much of the research work that Elie and Holden, co-founders of GiveWell, used to do, including all updates on current top charities, reviewing top charity contenders, managing research staff, and some intervention assessments. Holden now spends almost no time on GiveWell research.

We feel that we have sufficient capacity to follow up with our current top charities, consider promising contenders for top charity recommendations, and make decisions about Incubation Grants. We do not yet have sufficient capacity for reviewing the evidence for and modeling cost-effectiveness of interventions. We aim to make at least one hire for this work in the next few months. More on this in our post about our research plans for the year.

Outreach: As noted above, we feel we’ve reached the point where we are identifying outstanding giving opportunities more quickly than we can expand our reach to donors to fill the opportunities. Throughout most of our history, we felt that the opposite was true, that the amount of funding we could influence surpassed the opportunities we had identified, so this represents a significant shift for us. We don’t yet have concrete plans for future outreach work, but expect to give outreach significantly more attention than we have in the past.

We currently have one staff member, Catherine Hollander, who works on outreach full-time. Our outreach priorities in 2016 were to speak or meet with all major donors who were interested in talking to us, take any opportunities that came up to discuss our work with the media, and continue posting regularly to our blog. We feel that we accomplished our goals for connecting with major donors and keeping up with media requests, and fell short on blogging.

Catherine is leading the search for a Research Analyst, Outreach Focus to do more of the types of outreach we’ve focused on in the past, namely connecting with more media and major donors, and increasing the frequency of blog posts.

Does our impact justify our operating expenses?

GiveWell’s impact on donations (or “money moved”) to our recommended charities likely decreased somewhat in 2016. We are in the process of gathering and analyzing data on our influence on donations, but expect it to be in the range of $80-90 million to recommended charities and $9.2 million for Incubation Grants. Money moved to top charities in 2015 was $110 million.

Good Ventures’ giving to top charities fell from about $70 million to $50 million, due to changes in the way it is allocating funding across priorities and to a large one-off grant to GiveDirectly in 2015. Based on GiveWell’s recommendations, Good Ventures also funded $9.2 million in Incubation Grants, up from about $400,000 to $500,000 in each of 2014 and 2015.

Over the same period, we spent approximately $2 million on our operations. In total, GiveWell as an entity spent about $5.5 million on operational expenses, of which $3.5 million was spent on the Open Philanthropy Project.

We previously wrote that we believe that expenses that are 15% of money moved are well within the range of normal, so we feel comfortable with the relative size of our operating expenses at this point.

Does GiveWell have a positive and accurate public image?

We believe that GiveWell’s public image is largely positive and reasonably accurate. This is true for all or nearly all of the major media coverage we have received. See, for example, coverage on NPR and in The Atlantic, Esquire and Vox.

There are two aspects of our public image that we would like to change. First, media has sometimes portrayed our top charities as having guaranteed impact and as being the “best” charities—for example, a 2015 article in The Atlantic said, “If what you want is to save lives with certainty, several people said, you have to go to GiveWell.” We believe that our top charities offer the highest expected value among evidence-backed opportunities that we have found to date, but are not risk-free and may not be the best giving opportunities for donors with different values or unique expertise, connections, or resources. Second, charities may have an inaccurate view of the costs and benefits of engaging with us—more in this post.

Our biggest public image project in the last year was launching a redesigned website. This project took much longer than expected. The original launch date was April 2015, but due to unexpected problems and lack of staff capacity, it didn’t go live until September 2016. Our previous website had an outdated look and confusing architecture. We think the new one is a large improvement, though we aim to make some further improvements in the future.

Are we in a stable financial position?

The short answer is yes.

In 2016, we raised about $3 million in revenue available for funding our operations that was not specifically for funding Open Philanthropy Project expenses (Open Philanthropy has, recently, been fully funded by Good Ventures). We have roughly projected GiveWell’s expenses (excluding pre-split Open Philanthropy expenses) at $2.7 million in 2017 and $3.2 million in 2018. Given our money moved to top charities and our experiences with fundraising in the past, it seems reasonable to expect that we will be able to raise this funding, though we expect to do a more detailed analysis of our financial situation once the details of the split with Open Philanthropy have been fully worked out.

We do not expect revenue available for operations to decrease as a result of splitting with Open Philanthropy because most major donors have told us that they support GiveWell due to our work identifying top charities. We think it is likely that Good Ventures will continue to support 20% of GiveWell’s operational budget, as it has for the last several years.

The post GiveWell as an organization: progress in 2016 and plans for 2017 appeared first on The GiveWell Blog.

Natalie Crispin

GiveWell’s research plans for 2017

5 years 3 months ago

This is the second of four posts that form our annual review and plan for the following year. The first post reviewed our progress in 2016. The following two posts will cover GiveWell’s progress and plans as an organization and metrics on our influence on donations in 2016.

Our primary research goals for 2017 are to:

  • Speed up our output of new intervention assessments, by hiring a Senior Fellow and by improving our process for reviewing interventions at a shallow level.
  • Increase the number of promising charities that apply for our recommendation. Alternatively, we may learn why we have relatively few strong applicants and decide whether to change our process as a result. Research Analyst Chelsea Tabart will spend most of her time on this project.
  • Through GiveWell Incubation Grants, fund projects that may lead to more top charity contenders in the future and consider grantees No Lean Season and Zusha! as potential 2017 top charities.
  • Further improve the robustness and usability of our cost-effectiveness model.
  • Improve our process for following the progress of current top charities to reduce staff time, while maintaining quality. We also have some specific goals (discussed below) with respect to answering open questions about current top charities.

We discuss each of these goals in more depth below.

Intervention assessments

Intervention assessments are key to our research process. We generally only consider recommending funding for programs that are implementing one of our priority programs (an exception is if an organization has done rigorous evaluation of its own program, though in practice we have found this to be very rare). In recent years, we have completed few intervention reports, which has limited our ability to consider new potential top charities. We plan to increase the rate at which we form views on interventions this year by:

  • Hiring a Senior Fellow (or possibly more than one). We expect a Senior Fellow to have a Ph.D. in economics, public health, or statistics or equivalent experience and to focus on in-depth evidence reviews and cost-effectiveness assessments of interventions that appear promising after a shallower investigation. In addition, Open Philanthropy Project Senior Advisor David Roodman may spend some more time on intervention related work.
  • Doing low-intensity research on a large number of promising interventions. We generally start with a two to four hour “quick intervention assessment,” and then prioritize interventions for a 20-30 hour “interim intervention report” (example). We don’t yet have a good sense of how many of these of these we will complete this year, because we’re unsure both about how much capacity we will have for this work and about how many promising interventions there will be at each step in the process.
  • Continuing to improve our systems for ensuring that we become aware of promising interventions and new relevant research as it becomes available. We expect to learn about additional interventions by tracking new research, particularly randomized controlled trials, in global health and development and by talking to select organizations about programs they run that they think we should look into.

Charity applications

In the past few years, we have been surprised by how little interest there has been from charities in applying for a GiveWell recommendation. Our impression is that for global health and development charities there are relatively few funders of our size: in 2015, we tracked $110 million given due to our research; we are in the process of compiling the data for 2016, but expect it to be in the range of $80-90 million to recommended charities. We would like to better understand whether we have failed to get the word out about the potential value we offer or communicate well about our process and charities’ likelihood of success, or, alternatively, whether charities are making well-informed decisions about their fit with our criteria. (More on why we think more charities should consider applying for a GiveWell recommendation in this post.)

This year, we have designated GiveWell Research Analyst Chelsea Tabart as charity liaison. Her role is to increase and improve our pipeline of top charity contenders by answering charities’ questions about our process and which program(s) they should apply with, encouraging promising organizations to apply, and, through these conversations, understanding what the barriers are to more charities applying.

We aim by the end of the year to have a stronger pipeline of charities applying, have confidence that we are not missing strong contenders, or understand how we should adjust our process in the future.

Incubation Grants

We made significant progress on Incubation Grants in 2016 and plan in 2017 to largely continue with ongoing engagements, while being open to new grantmaking opportunities that are brought to our attention.

Among early-to-mid stage grants, we plan to spend the most time on working with IDinsight and New Incentives (where our feedback is needed to move the projects forward), and a smaller amount of time on Results for Development and Charity Science: Health (where we are only following along with ongoing projects).

Another major priority will be following up on two later-stage grantees, No Lean Season and Zusha!, groups that are contenders for a top charity recommendation in 2017. For No Lean Season, a program run by Evidence Action, our main outstanding questions are whether the program will have room for more funding in 2018 and whether monitoring will be high quality as the program scales. We have similar questions about Zusha! and in addition are awaiting randomized controlled trial results that are expected later this year.

Cost-effectiveness model

We plan to continue making improvements to our cost-effectiveness model and the data it draws on (separate from adding new interventions to the model, which is part of the intervention report work discussed above). Projects we are currently prioritizing include:

  • Making it more straightforward to see how personal values are incorporated into the model and what the implications of those values are.
  • Revisiting the prevalence and intensity adjustment that we use to compare the average per-person impact of deworming in places that our top charities work to the locations where the studies that found long-term impact of deworming were conducted. More in this post.
  • Improving the insecticide-treated nets model by revisiting how it incorporates effects on adult mortality and adjustments for regions with different malaria burdens and changes in malaria burden over time.

Current top charities

Our goal this year is to maintain the quality of top charity updates while decreasing the amount of staff time we spend and we ask top charities to spend on this work. Below, we detail our plans for following up with each charity.

Deworm the World Initiative, GiveDirectly, and Schistosomiasis Control Initiative (SCI)

We have now followed these groups for several years and do not have major outstanding questions about them. We plan to ask for updates on financial information, monitoring results, and room for more funding and have regular phone calls with them to learn about operational changes that might lead us to ask additional questions.

Against Malaria Foundation (AMF)

We have two major outstanding questions about AMF that we hope to make progress on this year:

  1. Will AMF’s monitoring processes be high quality? We wrote about our concerns about AMF’s past monitoring last year and expect new information to be available this year.
  2. Going forward, AMF aims to fund larger distributions and commit funding further ahead of when a distribution is scheduled to occur than it has, for the most part, done in the past. Will this increase the extent to which AMF funds displace funds from other sources, or will there continue to be evidence that AMF’s funds are largely adding to the total number of nets distributed? More on this question in our review of AMF.

To help us make progress on these questions, we and AMF have agreed to have monthly calls to discuss questions we have about the monitoring AMF is producing and what AMF is learning about distributions it is considering funding. We will likely also seek out calls with AMF’s partner organizations to discuss these questions.

In order to estimate AMF’s room for more funding, we will seek out information on the location and size of funding gaps for mass net distribution campaigns from AMF, the African Leaders Malaria Alliance, and possibly other funders of nets. As we have in the past, we will use this information in conjunction with conversations with AMF about non-funding bottlenecks to its ability to fill various gaps.

The END Fund – deworming program

Compared with the charities we have recommended for several years, we have more open questions about the END Fund. The main questions we plan to seek more information on this year are:

  1. We have not yet seen monitoring on par with that from our other top charities from the END Fund. We expect results from coverage surveys from END Fund programs this year. Will these surveys be high quality and demonstrate that the END Fund is funding successful programs?
  2. We have not yet tried to compare the cost-effectiveness of the END Fund to our other top charities in our cost-effectiveness model. We will be seeking additional information from the END Fund about cost per treatment and baseline infection rates.
  3. Questions around room for more funding: the extent to which funding due to GiveWell’s recommendation increases the amount that the END Fund spends on deworming versus other programs, actual and projected revenue from other sources, and what deworming grantmaking opportunities the END Fund expects to have.

We visited the END Fund’s programs in Rwanda and Idjwi island, DRC in January 2017 and will publish notes and photos from our visit shortly.

Malaria Consortium – seasonal malaria chemoprevention program

As with the END Fund, we have more open questions about Malaria Consortium than we do for the charities we have recommended for several years. Our main priorities are:

  1. Further research on the evidence of effectiveness, cost-effectiveness, and potential downsides of seasonal malaria chemoprevention (SMC) (due to time constraints we have not yet completed a full intervention report, though we felt sufficiently confident in the intervention to recommend Malaria Consortium).
  2. Getting a better understanding of the methodology Malaria Consortium uses for estimating coverage rates.
  3. Completing a more in-depth room for more funding analysis for the program for 2018 than we did for 2017.

Malaria Consortium expects to have several new studies of its SMC programs to share in April 2017 (details).

We may visit a Malaria Consortium seasonal malaria chemoprevention program in summer 2017.

Sightsavers – deworming program

As with the END Fund and Malaria Consortium, we have more open questions about Sightsavers than we do for the charities we have recommended for several years. We expect to make limited progress this year because the first deworming mass drug administration funded with GiveWell-influenced funds is not expected to take place until September at the earliest and monitoring results aren’t expected until early 2018. Because Sightsavers has done fairly little deworming in the past year, we don’t expect to be able to learn much from its ongoing programs. Our main priorities for the year are:

  1. Getting more information from Sightsavers about baseline prevalence and intensity of worm infections in the areas it is working, to inform our cost-effectiveness analysis.
  2. Using Sightsavers’ budget for the projects and planned treatment numbers to improve our estimate of the cost per treatment – another input into our cost-effectiveness analysis. Our current cost per treatment estimate is very rough.
  3. Completing a room for more funding analysis for 2018.

Standout charities

Standout charities are groups that we have a large amount of information about and that meet some but not all of our criteria. Because we have not followed them closely over time, it is possible that they may now be a stronger fit (or that they no longer focus on the program we reviewed). We plan to have at least one phone call with each of these groups to discuss whether anything has changed that might lead us to reopen consideration of the organization as a potential top charity. Due to our focus on organizations that are most likely to become top charities, we don’t expect to make this work a priority beyond that.

The post GiveWell’s research plans for 2017 appeared first on The GiveWell Blog.

Natalie Crispin

GiveWell’s progress on research in 2016

5 years 3 months ago

This is the first of four posts that form our annual review and plan for the following year. This post reviews and evaluates last year’s progress on our work of finding and recommending evidence-based, thoroughly vetted charities that serve the global poor. The following three posts will cover our plans for GiveWell’s research in 2017, GiveWell’s progress and plans as an organization and metrics on our influence on donations in 2016.

Summary

We feel that 2016 was a highly successful year for GiveWell’s research. We accomplished or made significant progress on all of our top priorities and accomplished some of the goals we didn’t know if we would have time for. Our research output was greater than in any past year. We added three new top charities and two new standout charities. We made important progress on building the pipeline of future GiveWell top charities through our work on GiveWell Incubation Grants. The research team’s staff capacity has continued to increase and we expect output to continue to grow.

More subjectively, we feel that the quality of our research has continued to improve. Of particular note are our improved understanding of room for more funding for insecticide-treated nets and of the evidence for deworming.

We compare our annual output for 2012-2016 in this spreadsheet.

Our progress in 2016 relative to our plans

In early 2016, we laid out our goals for research in 2016. Below we discuss each goal in two of the categories we set out last year, “top priorities” and “other research we will undertake if we have the time to do so,” and what progress we made on each.

“Top priorities”

Supporting the development of potential future GiveWell top charities: making grants to organizations that could become top charity contenders in the future or supporting research that could lead to more organizations that are a strong fit with our criteria. […]

We investigated and recommended that Good Ventures make grants to five early stage projects: No Lean Season (migration assistance), New Incentives (conditional cash transfers), Zusha! (road safety), Charity Science: Health (immunizations), and Results for Development (childhood pneumonia treatment). We have published grant descriptions for most of these grants; three are forthcoming. We have also begun working closely with IDinsight to partner with charities that work on priority programs to strengthen their monitoring and evaluation to increase the chances that the charities meet our criteria in the future.

Considering additional funding for insecticide-treated nets [beyond the Against Malaria Foundation] […]

We spent relatively little time on this priority because (a) the Against Malaria Foundation (AMF) succeeded in signing agreements for several major distributions early in the year, increasing our estimate of its ability to absorb additional funds; and (b) our initial conversations with large funders of nets seemed unlikely to result in a top charity recommendation. At the end of the year, we found that the Against Malaria Foundation was able to absorb considerably more funding than we directed to it. This was in part due to Good Ventures’s shift away from continued growth in funding for GiveWell top charities. Separately, we also gained a stronger understanding of the size and nature of funding gaps for nets globally through country case studies and conversations to understand the global funding landscape.

Intervention prioritization: quick investigations on a large number of interventions with the goal of finding more priority programs. […]

This was a major priority for us and we made some progress, but not as much as we wanted to. The main things we did were (a) quick reviews (3-10 hours spent reviewing the evidence base) for ~30 programs, which ultimately led us to prioritize seasonal malaria chemoprevention (SMC) and recommend Malaria Consortium’s work on SMC; (b) publishing three “interim intervention reports” (on SMC, integrated community case management, and severe acute malnutrition) which have provided a template for mid-level assessments and enabled more staff to produce such reports in 2017 (one of which, on Sayana Press, has been published); and (c) writing a full intervention assessment of voluntary male medical circumcision, which is now one of our priority programs.

Current top charities: continuing to follow our current top charities and trying to answer our highest priority unanswered questions about these groups. […]

We answered our most important questions about the top charities we recommended in 2015. In particular, we had stronger answers at the end of the year on AMF’s progress at signing agreements, the quality of AMF’s monitoring, Schistosomiasis Control Initiative’s past spending and financial position, and had an overall much stronger understanding of Deworm the World Initiative, particularly its work in Kenya.

However, we feel that we spent too much time on this work in 2016. Three staff spent the majority of their time on top charity updates and, while following the progress and plans of our top charities is a crucial piece of GiveWell’s work, the value we got from this work felt out of proportion with the time spent. In 2017, we plan to have a single staff member do most of this work and expect it to take a half to two-thirds of a full-time job. Three other staff will spend a small portion of their time, totaling approximately the equivalent of one full-time job, on this work.

New evidence on deworming and bednets. The next round of follow up on a key deworming study is expected to be available later this year and could make a big difference to our view of deworming. We’re also looking more into the degree to which insecticide resistance may be reducing the impact of bednets. […]

We completed an evidence review on the impact of insecticide resistance on malaria control. David Roodman, Senior Advisor at the Open Philanthropy Project spent several months revisiting the evidence for deworming and summarizing his findings in two blog posts.

We wrote last year that we expected to see a new round of follow-up data on a key deworming study that could significantly affect our view of deworming. We have seen preliminary results and hope to get more complete results later this year and write about them at that time. We also decided to recommend a ~$1 million grant to support a more intensive 20-year follow-up to the Worms at Work study (writeup forthcoming).

“Other research we will undertake if we have the time to do so”

Micronutrient fortification charities. […]

We completed interim reviews of Food Fortification Initiative and Project Healthy Children and added them to our list of standout charities.

Neglected tropical disease (NTD) charities [and…] other organizations – if organizations apply for a recommendation and seem sufficiently promising, we will aim to review them.

Perhaps the most important development in 2016 came out of two secondary goals for 2016: continuing investigations of deworming programs (which led to recommending Sightsavers and the END Fund for their work on deworming) and continuing to be open to applications from charities (which led to recommending Malaria Consortium for its work on seasonal malaria chemoprevention).

At the same time, in the past few years, we have been surprised by how little interest there has been from charities in applying for a GiveWell recommendation. We have come to believe that charities may have misconceptions about our process or lack the information about whether they would be a fit for our criteria, and that this could be improved by us reaching out to more promising organizations and taking the time to understand the reasons why they have not applied in the past. We discuss what we are doing this year on charity outreach in the next post in this series.

Surgery charities. We have had several conversations with organizations that work on cataract surgery and we may reach out to organizations that work on obstetric fistula surgery. […]

In addition to the conversations with organizations that work on on cataract, we also spoke to several groups that work on obstetric fistula. We have not identified a group we would like to invite to apply. This work is now moving ahead primarily through our engagement with IDinsight to work with charities to strengthen their monitoring and evaluation.

Publishing research we largely completed in 2015: updates on standout charities (GAIN, IGN, and Living Goods), interim reviews of charities we began investigating in 2015 (Sightsavers, END Fund, and Project Healthy Children), and intervention reports (folic acid fortification, surgery for cataracts, trachoma and fistula, measles immunization campaigns, mass drug administration for lymphatic filariasis, and “Targeting the Ultra Poor”).

We made some progress on this goal. We published updates on standout charities (GAIN, Iodine Global Network, and Living Goods), reviews of charities we began investigating in 2015 (Sightsavers, the END Fund, and Project Healthy Children), and one of the intervention reports we hoped to publish in 2016 (cataract surgery). We did not publish the other intervention reports we hoped to (folic acid fortification, surgery for trachoma and fistula, measles immunization campaigns, mass drug administration for lymphatic filariasis, and “Targeting the Ultra Poor”).

Other work

Compared to previous years, we made a lot of progress on improving the usability of and getting staff engagement with our cost-effectiveness model. We designated a staff member, currently Chris Smith, to work on this close to full-time. Chris re-formatted the file to make it easier for staff and others to input uncertain and subjective values, integrated the seasonal malaria chemoprevention analysis into the model, and restructured the bed nets model to take into account country-level variation in malaria rates.

The post GiveWell’s progress on research in 2016 appeared first on The GiveWell Blog.

Natalie Crispin
Checked
34 minutes 38 seconds ago
Exploring how to get real change for your dollar.