Evaluation of Givewell Blogs

Each spring, we share our plans for the year. Here, we highlight the work we plan to do in 2020 that is most likely to help us realize our mission of identifying and directing funding to highly cost-effective giving opportunities.1This post does not include a complete accounting of everything we plan to do in 2020. In particular, it does not include work aimed at primarily internal-facing results, such as improvements to internal staff communications. jQuery("#footnote_plugin_tooltip_1").tooltip({ tip: "#footnote_plugin_tooltip_text_1", tipClass: "footnote_tooltip", effect: "fade", fadeOutSpeed: 100, predelay: 400, position: "top right", relative: true, offset: [10, 10] }); We focus on three projects:

1. Expanding into new areas of research.
2. Searching for new, cost-effective funding opportunities in our traditional research areas.
3. Building our donor community.

Sharing our annual plans and publicly reflecting back on them a year later is our typical practice. This year, of course, is atypical. The plans we laid out internally at the beginning of the year have been disrupted by the COVID-19 pandemic. The plans we share in this post take the pandemic into account, but we are more uncertain than usual about what will happen in 2020. We expect that much of our work will go forward as anticipated, but we will be flexible if there are unforeseen disruptions or changes to our research agenda that result from the pandemic.

Expanding into new areas of research

Grants in response to the pandemic

We have already expanded into a new area of work in 2020: grantmaking in response to the COVID-19 pandemic. We don’t typically focus on high-uncertainty, short-timeline reviews of funding opportunities. However, we think that we should be open to making grants in a lower-information environment due to the potentially severe consequences of the pandemic in low- and middle-income countries, where we focus our work, and that acting sooner may be more impactful in preventing the spread of the disease. As of the publication of this post, we’ve made three grants for COVID-19 mitigation.

We plan to consider whether there are additional grants we should make in response to the pandemic. We will make these grants if we believe they are more cost-effective than the opportunities to which we would otherwise direct funds.

Prioritizing within public health regulation

We began this year with the goal of clarifying which areas were most promising within public health regulation, a relatively new-to-GiveWell domain that we see as potentially highly cost-effective but that we made limited progress in assessing last year.

We made a lot of headway on this work in the early months of 2020. We conducted research that led us to prioritize alcohol policy and pesticide suicide prevention and to deprioritize additional work on other areas such as air pollution, tobacco, and road safety. Shortly before the pandemic intensified, we also began an investigation into a potential alcohol policy grant we might make. We were delayed in completing our investigation of the alcohol grant because the potential grantee shifted its focus to COVID-19 response.

We have paused work on public health regulation for now, outside of an investigation into a potential grant renewal for the 2017 GiveWell Incubation Grant recipient Centre for Pesticide Suicide Prevention. We plan to do more work on public health regulation in the coming months or after 2020, depending on our capacity for work outside of assessing opportunities in response to COVID-19. When we return to public health regulation, we plan to complete our investigation into the alcohol policy grant mentioned above. At that point, we plan to reflect on what we’ve learned to date about public health regulation before deciding whether to do additional work in this area.

Searching for new, cost-effective funding opportunities in our traditional research areas

We continue to assess evidence-backed global health and poverty alleviation programs that may meet our traditional criteria. We significantly expanded our research team in 2019 and expect to make great progress in vetting new evidence and moving programs through our review process in 2020.

Our research process operates as a funnel: we start by conducting shallow reviews of a large number of programs, followed by in-depth reviews of the most promising. This year, we’re planning to assess a large number of programs at various stages of the funnel. Here, we highlight a few that we’re particularly excited about:

• New Incentives. We recently received preliminary results from a randomized controlled trial (RCT) of New Incentives‘ work. New Incentives provides cash incentives for caregivers in North West Nigeria who take their babies to a health clinic to receive routine immunizations. New Incentives is a GiveWell Incubation Grant recipient and the study of its work, which was funded by another GiveWell Incubation Grant, is the first RCT into which GiveWell has had significant input. This year, we expect to complete our review of the RCT results and to make a decision about whether to direct additional funding to support New Incentives’ work.
• Evidence Action programs. We plan to complete additional research on two programs run by Evidence Action: GiveWell standout charity Dispensers for Safe Water and a maternal syphilis screening and treatment program.

We hope to conduct in-depth reviews of one to two other charities (most likely Precision Agriculture for Development and/or Sanku, though possibly others) as well, but may have limited capacity to do so due to COVID-19-responsive work.

Building the GiveWell donor community

We aim to grow the GiveWell donor community in 2020.

Our marketing team plans to reach new donors through advertising campaigns, including additional advertising on podcasts. The marketing team also plans to conduct and support research to inform updates to our website that may improve how we present information. For example, the team is examining how users engage with our website and where they may become confused or misinterpret our work, so that we can make improvements to those areas.

We’re hiring!

We have ambitious plans this year and are looking to hire staff to help us achieve our goals. If you think the plans above sound exciting, we encourage you to consider whether working at GiveWell would be a good fit for you. Many of our successful hires in the past have come through the community of people who engage with our work. You can read about the roles we’re hiring for here. Staff may work remotely and we accommodate flexible work schedules, including flexible hours. Additional details are in each job description.

We hope you’ll consider joining the team and sharing this with others who might like to. Thank you!

Notes   [ + ]

1. ↑ This post does not include a complete accounting of everything we plan to do in 2020. In particular, it does not include work aimed at primarily internal-facing results, such as improvements to internal staff communications. function footnote_expand_reference_container() { jQuery("#footnote_references_container").show(); jQuery("#footnote_reference_container_collapse_button").text("-"); } function footnote_collapse_reference_container() { jQuery("#footnote_references_container").hide(); jQuery("#footnote_reference_container_collapse_button").text("+"); } function footnote_expand_collapse_reference_container() { if (jQuery("#footnote_references_container").is(":hidden")) { footnote_expand_reference_container(); } else { footnote_collapse_reference_container(); } } function footnote_moveToAnchor(p_str_TargetID) { footnote_expand_reference_container(); var l_obj_Target = jQuery("#" + p_str_TargetID); if(l_obj_Target.length) { jQuery('html, body').animate({ scrollTop: l_obj_Target.offset().top - window.innerHeight/2 }, 1000); } }

The post GiveWell’s plans for 2020 appeared first on The GiveWell Blog.

GiveWell grew significantly in 2019. We hired 13 full-time staff members, bringing our total size to 37, and expanded our ability to take on new projects across domains. We feel positioned to do more and better work going forward as a result.

We see a strong indication that the amount of funding we directed to our recommended charities increased last year, too. While we haven’t reconciled all giving from 2019, the value of donations we processed increased by about 30% in 2019.

We’re proud of what we accomplished in 2019. We also fell short of some goals last year. Most notably, we failed to make as much progress as we planned in researching new areas of global health and poverty alleviation.

This blog post provides a brief look at our key successes and failures last year. A more detailed accounting of how our progress in 2019 compared to the goals we set is available on this page.

Successes

Hiring new staff

Years of planning for our needs and recruiting efforts culminated in hiring 13 new staff to join our small team in 2019. GiveWell ended last year over 50% bigger than it was at the end of 2018.

We hired across the domains of our work: seven joined the research team, two joined the outreach team, three joined the operations team, and one will serve as Managing Director. We expect each staff member will enable us to accomplish more and achieve better outcomes across these key areas of our work. Below, we highlight a few senior hires whom we expect to help steer the direction of our work.

• Managing Director. We hired Neil Buddy Shah as our first Managing Director in late 2019. Buddy will work closely with GiveWell’s CEO Elie Hassenfeld to set GiveWell’s high-level strategy. He will also engage with the international development community to learn from and contribute to discussions of how to do as much good as possible and to identify promising funding opportunities. We expect Buddy to start at GiveWell this summer.
• Research. We’ve been looking to hire senior researchers to expand our ability to assess new types of evidence since 2016. We described in early 2019 how our research is evolving and how we hoped to hire additional experienced researchers to enable us to do this work. We hired Alex Cohen and Teryn Mattox as Senior Fellows in 2019.
• Outreach. We’ve gradually been expanding our outreach work since 2017. In 2019, Steph Stojanovic and Jim Bobowski joined GiveWell as a Major Gifts Officer and VP of Marketing, respectively, and will lead our future work on donor retention and acquisition.
• Operations. Our Director of Operations, Whitney Shinkle, who joined GiveWell in 2018, built out her team by bringing on three new staff members in 2019.

GiveWell’s expansion enables us to improve in each area of our work. For example, we hope to make progress this year on evaluating new research areas as well as looking for new, cost-effective room for more funding in the areas in which we’ve traditionally worked. We also expect to increase the amount of money we direct to our recommended charities with our expanded outreach team.

Increasing the amount of funding we direct to recommended charities

We see an early indication that we directed more money to our recommended charities in 2019 than we did the previous year: the value of donations we processed in 2019 grew by around 30%.

Directing funding to our recommended charities is one of the main metrics to which we hold ourselves accountable. We aim to grow the amount of money we move to our top charities each year.

The “donations we processed” figure doesn’t account for the donations that were made through our partner organizations and directly to our recommended charities, but we think it’s a good sign of our continued influence last year. We’ll share the final figure later this year, once we have more information.

Shortcomings

Making progress in understanding new areas of research

In May 2019, we wrote that we planned to make progress on exploring new areas of research in 2019. In particular, we planned to focus on public health regulation and ways to support government aid agencies.1“We plan to look into several new areas in 2019, including public health regulation and possible paths to support government aid agencies…. We also plan to continue our investigation into possible paths to support government aid agencies; in particular, we plan to complete an investigation into an opportunity to do so in the area of results-based financing.” GiveWell blog, GiveWell’s plans for 2019, May 16, 2019 jQuery("#footnote_plugin_tooltip_1").tooltip({ tip: "#footnote_plugin_tooltip_text_1", tipClass: "footnote_tooltip", effect: "fade", fadeOutSpeed: 100, predelay: 400, position: "top right", relative: true, offset: [10, 10] }); Due to shifting responsibilities among the research team, we did not make progress in this area last year.

We had planned for a single staff member to focus on this work. As this person’s responsibilities evolved throughout the year, they were unable to dedicate significant time to this project. This led us to deprioritize making progress in this area.

Conclusion

The above highlights don’t tell the full story of what we achieved—we had a busy year! Listing everything we worked on in 2019 would lead to a very long post. But, to briefly share a few other interesting projects (this list is non-exhaustive):

• We published research on moral weights. This was the culmination of over two years of research and will inform and improve our ability to make difficult tradeoffs in our funding decisions going forward.
• We recommended a $1 million Incubation Grant to support the work of Fortify Health on whole wheat flour iron fortification in India. We believe Fortify Health, a young organization, may one day become a GiveWell top charity.
• Four staff members visited Malaria Consortium, our current top recommendation for donors, to deepen our understanding of its seasonal malaria chemoprevention work in Burkina Faso.

We’re proud of what we accomplished last year and look forward to sharing our plans for 2020 in a forthcoming post. Additional details on our 2019 goals and progress are available here.

Notes   [ + ]

1. ↑ “We plan to look into several new areas in 2019, including public health regulation and possible paths to support government aid agencies…. We also plan to continue our investigation into possible paths to support government aid agencies; in particular, we plan to complete an investigation into an opportunity to do so in the area of results-based financing.” GiveWell blog, GiveWell’s plans for 2019, May 16, 2019 function footnote_expand_reference_container() { jQuery("#footnote_references_container").show(); jQuery("#footnote_reference_container_collapse_button").text("-"); } function footnote_collapse_reference_container() { jQuery("#footnote_references_container").hide(); jQuery("#footnote_reference_container_collapse_button").text("+"); } function footnote_expand_collapse_reference_container() { if (jQuery("#footnote_references_container").is(":hidden")) { footnote_expand_reference_container(); } else { footnote_collapse_reference_container(); } } function footnote_moveToAnchor(p_str_TargetID) { footnote_expand_reference_container(); var l_obj_Target = jQuery("#" + p_str_TargetID); if(l_obj_Target.length) { jQuery('html, body').animate({ scrollTop: l_obj_Target.offset().top - window.innerHeight/2 }, 1000); } }

The post Reflecting on our progress in 2019 appeared first on The GiveWell Blog.

We see an early indication that GiveWell continued its trajectory of robust donor growth last year. The total value of donations processed by GiveWell increased 30% in 2019.[1]

We’re sharing this data now because we believe it is an informative early update about our growth last year. However, GiveWell-processed donations don’t tell the full story of our impact. Many donors who rely on our research give via our partner organizations or directly to our top charities. Their gifts account for the majority of donations due to our work and are not processed by GiveWell. Information about these gifts is time-consuming to gather and has usually led us to release our metrics data many months after the end of the year. We plan to release a complete 2019 metrics report and assessment of our impact, including donations not processed by GiveWell, later this year.

Here’s what we know so far, based on the nearly complete information we have about donations we processed:

• We processed $54.1 million in donations in 2019. Sixty-five percent of this amount was restricted to our recommended charities and 35% was unrestricted, which we may use to support GiveWell’s operations.
• Support from donors giving $10,000 to $100,000 comprised the largest proportion of our growth (35%).[2]
• Returning donors who gave more than last year made up 75% of our growth in funds donated (excluding anonymous donations).[3]
• We believe that the majority of our growth was organic and would have occurred without any outreach and marketing efforts from GiveWell, although we can attribute some to specific outreach and marketing initiatives.[4]

We’re encouraged by this growth and excited to write about it. We also discuss below some ways that GiveWell-processed donations could be a misleading indicator of our overall impact.

Summary

This post covers the following topics:

• About GiveWell-processed donations (More)
• Donations we processed in 2019
  • Restricted donations by donor size (More)
  • Unrestricted donations by donor size (More)
  • New donors and returning donors (More)
• Ways in which these data are incomplete (More)
• Additional information: Where did donors give? (More)

About GiveWell-processed donations

This is our first time releasing information on GiveWell-processed donations as a standalone post.

It is very time-consuming to gather and analyze information from the many disparate sources through which GiveWell donors can support GiveWell and our recommended charities.[5] We have generally shared reports on donations due to our work quite late in the year as a result.[6]

We do have nearly complete information at this point of the year on GiveWell-processed donations in 2019. We’re experimenting with sharing this information as an early look at our impact.

What is included in GiveWell-processed donations?

The $54.1 million only includes donations that we processed ourselves, such as gifts that were made by credit card on our website, bank transfers to GiveWell’s account, checks made out to GiveWell, and other options listed here.[7] It includes both restricted and unrestricted donations:

• Restricted: A donor tells us to use the gift for one or more of our top charities or for “Grants to recommended charities at GiveWell’s discretion,” which we allocate quarterly to the recommended charity or charities with the most pressing needs.
• Unrestricted: A donor does not place any restrictions on the donation. We may use the donation for GiveWell’s operations or grant it to charities.

The $54.1 million does not include gifts that were made directly to our recommended charities or via our partner organizations.[8] Additional ways in which GiveWell-processed donations may not tell the full story of our impact are discussed below.

Our impact: donations we processed

Restricted donations: by donor size

We saw strong growth in 2019 in restricted donations across donor size categories under $1 million.[9] Most donors making restricted donations of $1 million or more give directly to our top charities and not through GiveWell; we discuss this in greater detail below.

Note: Anonymous donations are only tracked in the penultimate row and are not reflected in the other size category totals.[10]

Note: This table excludes all anonymous donors.

Unrestricted donations: by donor size

The largest absolute growth (just over $1 million) and relative growth (69%) in unrestricted donations was from donors giving between $100,000 and $1 million. We explain the slight decline in unrestricted donations from donors giving more than $1 million in the following footnote.[11]

Note: Anonymous donations are only tracked in the penultimate row and are not reflected in the other size category totals.

The total number of non-anonymous donors providing unrestricted gifts increased by 22%, with the strongest relative growth (34%) in the number of donors giving between $1,000 and $10,000.

Note: This table excludes all anonymous donors.

New donors and returning donors

Returning donors increasing their donations drove approximately 75% of our growth in funds donated, excluding anonymous donations. Funds from new donors increased more than 40% over the previous year and accounted for approximately 25% of our total growth in non-anonymous funds donated.

Note: This chart shows total donations by new and returning donors, as well as from anonymous donors who cannot be included in either category, as a percentage of all donations (restricted and unrestricted).

How GiveWell-processed donations don’t tell the full story

While we see GiveWell-processed donations as an early indicator of strong growth, we have significant uncertainty about the final picture of our influence in 2019.

Donors giving over $1 million have tended to donate directly to our recommended charities rather than through GiveWell, and so are not well-represented in GiveWell-processed donations.[12] They have also accounted for a large proportion of our impact each year as major contributors to our recommended charities. This group made up roughly one-third of our money moved to recommended charities in 2018, excluding donations from Open Philanthropy.[13] Our best guess is that growth among donors giving under $1 million is related to growth among donors giving over $1 million—if more donors are giving to GiveWell in general, we think it’s likely that additional larger donors will find our work. However, we’re unsure of the extent to which 2019 donors who gave under $1 million are predictive of the 2019 donors who gave over $1 million directly to our recommended charities.

Another way in which GiveWell-processed donations could fail to predict our overall growth is if donors shifted how they gave in 2019. If a greater proportion of donors chose to give through GiveWell in 2019, extrapolating from the proportion of GiveWell-processed donations in previous years to our total impact in 2019 would be overly optimistic. Although we don’t expect this to be the case, we won’t know for sure until we see the complete data from 2019.

Additional information: Where did donors give in 2019?

Restricted gifts accounted for $35 million of the $54.1 million we processed last year. Donors allocated 64% of restricted dollars in 2019 to “Grants to recommended charities at GiveWell’s discretion,” which was (and is) our top recommendation for GiveWell donors.

The remainder of the $54.1 million ($19.1 million) was unrestricted. We typically use unrestricted funding for our operations, though some of these funds will likely be granted to charities.[14]

Footnotes

Footnotes for this post may be found here.

—
Devin Jacob co-authored this post.

The post How did we do in 2019? A preliminary look at our growth. appeared first on The GiveWell Blog.

GiveWell is dedicated to finding outstanding giving opportunities and publishing the full details of our analysis. In addition to evaluations of other charities, we publish substantial evaluation of our own work. This post lays out highlights from our 2018 metrics report, which reviews what we know about how our research impacted donors. Please note:

• We report on “metrics years” that run from February through January; for example, our 2018 data cover February 1, 2018 through January 31, 2019.
• In an effort to present a more comprehensive measure of our influence on charitable giving, this year’s metrics report includes GiveWell Incubation Grants in our headline “money moved” figure. In previous reports we have excluded Incubation Grants from this figure.

Summary of influence: In 2018, GiveWell influenced charitable giving in several ways. The following table summarizes our understanding of this influence.

Headline money moved: In 2018, we tracked $141 million in money moved to our recommended charities and via our Incubation Grants program. Our money moved only includes donations that we are confident were influenced by our recommendations.

Money moved by charity: Our nine top charities, including Evidence Action’s No Lean Season which is no longer a top charity, received the majority of our money moved. Our eight standout charities received a total of $2.7 million. We also tracked $0.5 million in unrestricted donations to organizations that run programs we recommend and $15.9 million in Incubation Grants.

Money moved by size of donor: We have less reliable data on individual donors in 2018 than in previous years, primarily as a result of new European privacy regulations that have led some of our recommended charities to share less detailed data with us. We have estimated the number of donors in each size category we track. We estimate that the number of donors in most categories was roughly unchanged compared to 2017 and that the amount of money in most categories increased slightly. Notably, we saw a large increase in the amount donated from donors giving $1 million or more. In 2018, we estimate that 92% of our money moved (excluding Good Ventures) came from the 20% of our donors who gave $1,000 or more. Details on how we produced these estimates are available in the full report.

GiveWell’s expenses: GiveWell’s total operating expenses in 2018 were $4.3 million.

Donations supporting GiveWell’s operations: GiveWell raised $12.4 million in unrestricted funding (which we mostly use to support our operations) in 2018, compared to $5.7 million in 2017. The eleven largest individual donors, plus Good Ventures, contributed 67% of GiveWell’s operational funding in 2018.

Web traffic: The number of unique visitors to our website declined 22% in 2018 compared to 2017 (when excluding visitors driven by AdWords, Google’s online advertising product).

For more detail, see our full metrics report (PDF).

The post GiveWell’s money moved and web traffic in 2018 appeared first on The GiveWell Blog.

Our top priorities this year support our goals to (a) increase the impact per dollar of the funds we direct and (b) increase our money moved. In 2019, we are focused on:

• Building research capacity. (More)
• Experimenting with approaches to outreach to find ones that we can scalably use to drive additional money moved. (More)
• Exploring new areas of research. (More)
• Improving GiveWell’s organizational strength. (More)
• Ongoing research. (More)

Building research capacity

We announced earlier this year our plans to hire researchers at three levels of seniority, listed here from most junior to most senior: Research Analyst, Senior Research Analyst, and Senior Fellow. Our goal is to have 3-5 signed offer letters in hand from new research staff by the end of 2019.

We’re hoping that additional research capacity will enable us to expand the scope of GiveWell’s research, with the aim of finding opportunities that are more cost-effective than our current top charities. We’re planning to roughly double the size of the research team over the next few years.

Outreach experimentation

We plan to expand our outreach to current and potential donors going forward, with the aim of increasing the amount of money we direct to our recommended charities. As part of this work, we recently hired Stephanie Stojanovic as our first Major Gifts Officer. Our goal is to decide by the end of the 1st quarter of 2020 whether to scale our staff capacity further in the area of major gifts, based on Stephanie’s initial work.[1]

We’re also planning to conduct experiments in 2019 related to how we message about our work to reach more people. These experiments could include work on search engine optimization and building landing pages that aim to communicate what GiveWell does and why it’s valuable, among other possibilities. We expect to have results by early 2020, as the bulk of donations we receive are made in December.

Finally, we’re planning to search for a VP of Marketing to oversee work across outreach domains (including major gifts, donor retention, advertising, marketing, and written communications). We guess there is a 50 percent chance we make a hire for this role in 2019.

Exploring new areas of research

As mentioned above, we’re in the early stages of expanding the scope of GiveWell’s research. We plan to look into several new areas in 2019, including public health regulation and possible paths to support government aid agencies.

This work is new for GiveWell, and as noted in our 2018 review post, we failed to make as much progress as we hoped in 2018 on our work on public health regulation. In 2019, we’re aiming to get substantially closer to the point where we have the staff structure to support grantmaking in new areas, though given how early we are in this work, we don’t yet have concrete goals we’re confident that we’ll achieve.

A stretch goal for 2019 is to settle on a structure that we believe will support grantmaking in public health regulation and begin recommending grants in that area.

We also plan to continue our investigation into possible paths to support government aid agencies; in particular, we plan to complete an investigation into an opportunity to do so in the area of results-based financing.

Improving GiveWell’s organizational strength

We expect to need additional operations capacity to maintain critical functions as GiveWell grows and to improve the organization going forward. We plan to hire one Operations Associate this year to assist with general operations needs, such as improving HR practices.

We plan to hire many new staff over the coming years. In preparation, we plan to improve our procedures and information for recruiting, vetting, and onboarding staff to GiveWell this year, such as by improving inclusive recruitment practices and updating the substantive content of our onboarding activities. We also plan to improve our systems for soliciting feedback from staff about how GiveWell can improve as an organization, in order to give management better insight into how things are going.

To accommodate our planned expansion, we plan to move to a new office that better suits our expected size and staff requirements.

Ongoing research

We have a number of ongoing research projects, detailed here. These include:

• Completing a full draft of qualitative assessments of our top charities. In theory, we aim to maximize one thing with our top charity recommendations—total improvement in well-being per dollar spent—and this is what our cost-effectiveness estimates intend to capture. In practice, there are costs and benefits that we do not observe and are not estimated in our models, and so we allow for qualitative adjustments to affect our recommendations. We’re in the process of laying out a framework for qualitatively assessing relative organizational strength.
• Updating key inputs into our cost-effectiveness estimates, such as:
  • How we use vitamin A deficiency data.
  • Using new malaria prevalence and child mortality data.
  • Using new data to update our estimates of costs incurred and target population reached for five of our top charities: Malaria Consortium’s seasonal malaria chemoprevention program, the Against Malaria Foundation, Helen Keller International’s vitamin A supplementation program, Sightsavers’ deworming program, and Evidence Action’s Deworm the World Initiative.
  • Better understanding the counterfactual to the work Evidence Action’s Deworm the World Initiative has done in India. This goal is one we hope to achieve if we have time, but is not critical to our assessment.

Conclusion

The concrete goals we aim to achieve in 2019 follow. We plan to revisit this list in early 2020 to assess our progress relative to our expectations, and to publish a blog post accounting for our work:

• Building research capacity
  • Have 3-5 signed offer letters in hand from new research staff.
• Outreach experimentation
  • Decide whether to scale up Major Gifts work.
  • Conduct experiments related to messaging about our work to reach more people.
  • Complete search for a VP of Marketing.
• Exploring new areas of research
  • Look into several new areas, including public health regulation and possible paths to support government aid agencies. Get substantially closer to the point where we have the staff structure to support grantmaking in new areas.
  • Stretch goal: Settle on a structure that we believe will support grantmaking in public health regulation and begin recommending grants in that area.
• Improving GiveWell’s organizational strength
  • Hire one Operations Associate.
  • Improve the staff onboarding process at GiveWell.
  • Improve systems for soliciting feedback from staff.
  • Move to a new office.
• Ongoing research
  • Our full list of concrete research goals for 2019 is in this document.

Notes

1. The majority of donations in support of GiveWell’s recommended charities are made in the fourth quarter of the year, and we generally don’t have a clear sense of the total amount given to GiveWell directly until the first quarter of the following year (and the second quarter for direct-to-charity donations that are reported to us), so we think this is the right time frame on which to assess major gifts work.

The post GiveWell’s plans for 2019 appeared first on The GiveWell Blog.

2018 was a successful year for GiveWell. We achieved most of our goals and our money moved (donations made to our recommended charities due our research) increased significantly.

Each year, we look back at the goals we set the previous year and reflect on how our progress compared to our expectations.

This post will briefly discuss our key achievements and failures in 2018. We describe in detail our progress on the goals we outlined in 2018 here.

In 2018, we:

• Directed an estimated $65 million in donations to our top charities, not including the contributions of Good Ventures, a large foundation with which we work closely.
• Added senior hires in operations and outreach: a Director of Operations (Whitney Shinkle) and Head of Growth (Ben Bateman). We expect Whitney and Ben to make major contributions to our work in these domains.
• Continued to improve and expand our core research product, completing new intervention reports, deepening our analysis for several key inputs into our cost-effectiveness model, and providing more transparent explanations for how we decided to allocate funds between top charities.

Key achievements

Donations made to top charities as a result of our research

We currently estimate that the amount of money we directed to our top charities in 2018 was more than $65 million, not including the contributions from Good Ventures, a large foundation with which we work closely. This represents an increase of more than $20 million over 2017. The increase largely came from two multi-million dollar donations from donors who had supported GiveWell and/or our recommended charities in the past.

We plan to publish a full report on our 2018 donations and web traffic shortly.

Outreach and operations

We made two key senior hires in 2018: (1) Whitney Shinkle, who joined us in April as our new Director of Operations, and (2) Ben Bateman, who joined us in June as our first-ever Head of Growth.

We expect Whitney and Ben to play critical roles in laying the foundation to increase the amount of funding we can direct to our top charities. Whitney’s team, for example, is responsible for processing donations to our recommended charities, and for preparing GiveWell to increase the size of its staff. Ben is leading experiments to evaluate different ways we might increase the amount of funding we direct to our top charities via marketing and outreach.

Full details of our performance against our 2018 outreach and operations goals are here.

Research

We completed several projects that improved the quality of our cost-effectiveness estimates and how we write about them, and that we believe led to better decisions about where to allocate funds. For example, we made a major change to how we calculate worm intensity in the areas where our top charities work.

We also improved our transparency about these decisions, breaking our blog posts announcing our top charities into component parts to make them easier to follow (see 1, 2, and 3) and delving into more detail on our principles and funding gap analyses.

We published five new intervention reports, two of which were on the evidence for community-based management of acute malnutrition and syphilis screening and treatment during pregnancy, and recommended five new GiveWell Incubation Grants and two grant renewals. Two of our new grants supported Evidence Action Beta’s incubator and J-PAL’s Innovation in Government Initiative, respectively.

Full details of our performance on our 2018 research goals are here.

Key failures

Outreach and operations

We took a number of steps to improve our outreach to GiveWell’s existing donors. We had hoped this would lead to material improvements in retention of our donors as well as the amount of funding we were able to direct to our top charities from our donors. We haven’t completed a careful assessment of this work, but our belief at this point is that the steps we took last year are unlikely to have had a significant impact on donor retention.

Research

We made relatively little progress in exploring new areas of research (i.e., policy-oriented causes).

Additional information

This page has more details on our progress toward the goals we laid out in early 2018.

We plan to publish a post soon detailing our plans for 2019.

The post Review of GiveWell’s work in 2018 appeared first on The GiveWell Blog.

To date, most of GiveWell’s research capacity has focused on finding the most impactful programs among those whose results can be rigorously measured. This work has led us to recommend, and direct several hundred million dollars to, charities improving health, saving lives, and increasing income in low-income countries.

One of the most important reasons we have focused on programs where robust measurement is possible is because this approach largely does not rely on subject-matter expertise. When Holden and I started GiveWell, neither of us had any experience in philanthropy, so we looked for charities that we could evaluate through data and evidence that we could analyze, to make recommendations that we could fully explain. This led us to focus on organizations that had impacts that were relatively easy to measure.

The output of this process is reflected in our current top charities and the programs they run, which are analyzed in our intervention reports.

GiveWell has now been doing research to find the best giving opportunities in global health and development for 11 years, and we plan to increase the scope of giving opportunities we consider. We plan to expand our research team and scope in order to determine whether there are giving opportunities in global health and development that are more cost-effective than those we have identified to date.

We expect this expansion of our work to take us in a number of new directions, some of which we have begun to explore over the past few years. We have considered, in a few cases, the impact our top and standout charities have through providing technical assistance (for example, Deworm the World and Project Healthy Children), supported work to change government policies through our Incubation Grants program (for example, grants to the Centre for Pesticide Suicide Prevention and Innovation in Government Initiative), and begun to explore areas like tobacco policy and lead paint elimination.

Over the next several years, we plan to consider everything that we believe could be among the most cost-effective (broadly defined) giving opportunities in global health and development. This includes more comprehensively reviewing direct interventions in sectors where impacts are more difficult to measure, investigating opportunities to influence government policy, as well as other areas.

Making progress in areas where it is harder to determine causality will be challenging. In my opinion, we are excellent evaluators of empirical research, but we have yet to demonstrate the ability to make good judgments about giving opportunities when less empirical information is available. Our values, intellectual framework, culture, and the quality of our staff make me optimistic about our chances, but all of us at GiveWell recognize the difficulty of the project we are embarking on.

Our staff does not currently have the capacity or the capabilities to make enough progress in this direction, so we are planning to significantly increase the size of our staff. We have a research team of ten people, and we are planning to more than double in size over the next three years. We are planning to add some junior staff but are primarily aiming to hire people with relevant experience who can contribute as researchers and/or managers on our team.

GiveWell’s top charities list is not going to change dramatically in the near future, and it may always include the charities we recommend today. Our top charities achieve outstanding, cost-effective results, and we believe they are some of the best giving opportunities in global health and development. We expect to conclude that many of the opportunities we consider in areas that are new for us are less cost-effective than those we currently recommend, but we also think it is possible that we will identify some opportunities that are much more cost-effective. We believe it is worth a major effort to find out.

What areas will we look into?

As with any exploration into a new area, we expect the specifics of the work we will undertake to shift as we learn more. Below we discuss two major areas of work we are embarking on and building our team for currently. In the long term, we are open to considering making grants or recommendations in all areas of global health and development. We have not yet comprehensively considered what those areas might be, but they could include (for example) research and development, or social entrepreneurship.

Using reasoned judgment and less robust evidence to come to conclusions about additional direct-delivery interventions

In the past, we have often asked, “does this intervention meet our criteria?” rather than “what is our best guess about how promising this intervention is relative to our top charities?” Our intervention report on education is a good example of asking the question, “does this meet our criteria?” It reviews all randomized controlled trials of education programs that measure long-term outcomes, but it does not attempt to reach a bottom line about how cost-effective education in developing countries is.

We plan to more deeply explore how we can reach conclusions about how areas such as nutrition, agriculture, education, reproductive health, surgical interventions, mental health, and non-communicable diseases compare to our current top charities.

Investigating opportunities to improve government spending and influence government policy

Some of the areas we will consider exploring to leverage government resources and affect government policy are:

Broad thematic area Examples Brief rationale Public health regulation Tobacco control; lead paint regulation; road traffic safety; air pollution regulation; micronutrient fortification and biofortification; sugar control; salt control; trans-fats control; legislation to reduce counterfeit drugs; soil pollution; pesticide regulation; occupational safety laws Some regulatory interventions to improve public health have had a large impact in high-income countries. Low-income countries can lack the government capacity or political will to implement these regulations. Charities can advocate or provide technical assistance to accelerate regulation and improve implementation. Improving government program selection Innovation in Government Initiative; Innovations for Poverty Action; IDInsight; Center for Effective Global Action Low-income country governments may not have the capabilities to select good programs to support with their limited budgets. Charities can directly assist governments to make better decisions in the short term, or help improve their capabilities to do so independently over the longer term. Improving government implementation Results for Development; Deworm the World in India Low-income countries may not have the capabilities to implement programs effectively. Charities can directly assist governments to improve the reach or quality of programs in the short term, or help improve their capabilities to do so independently over the longer term. Improving non-programmatic government capabilities Building State Capability Improving the administrative capabilities of a government can result in broad improvements in the way countries function. Improved or increased aid spending Center for Global Development; ONE Campaign; Overseas Development Institute; Brookings Institution Spending by high-income countries on global health and development accounts for a large portion of total spending in this area. There are groups who advocate for, and provide technical assistance to improve aid spending. Advocating for increased spending on highly cost-effective, direct-delivery programs Malaria No More; Uniting to Combat Neglected Tropical Diseases GiveWell’s money moved is a small proportion of total global spending on aid. We believe these dollars would go further if a portion were redirected to the highly cost-effective, direct-delivery programs we recommend. Increasing economic growth and redistribution Charter cities; infrastructure programs; trade liberalization; macroeconomic policy; International Growth Centre; tax reform Economic growth is an important driver of economic well-being over the long term. Government policies can be an important determinant of the rate of economic growth and the degree to which growth translates into well-being for the population. There may be opportunities for charities to assist in promoting growth and better distributional outcomes. Negative externalities of high-income country policies Immigration reform; trade liberalization; reducing carbon emissions Governments of high-income countries are incentivized to select policies which are popular with their own voters. These policies can impose substantial costs on low-income countries. Charities can advocate for these policies to be changed. Improving governance Election monitoring; anti-corruption; good governance awards; term limits; peace programs There are particular characteristics of the governance of a country (e.g. democratic accountability, stability, human rights, lack of corruption) which are strongly associated with the well-being of its people. Charities can advocate for these characteristics to be adopted or strengthened. Reducing the cost of health commodities Clinton Health Access Initiative Reductions in the cost of medical commodities can result in improved coverage and improved economic well-being for low-income households. Improving data collection Institute for Health Metrics and Evaluation Improved data can be used by a variety of actors to make better decisions. One-off big bets Mosquito gene drives advocacy and research We may come across promising projects that do not fit neatly into one of the above categories. How will our analysis change? How will it be the same?

Writing up and publishing the details of the reasoning behind the recommendations we make is a core part of GiveWell. We will remain fully transparent about our research.

Judgment calls that are not easily grounded in empirical data have long been a part of GiveWell’s research. For example, we make difficult, decision-relevant judgment calls about moral weights, interpreting conflicting evidence about deworming, and estimating the crowding-out and crowding-in effects of our donations on other actors (what we call leverage and funging).

As we move into areas where measuring outcomes and attributing causal impact is more difficult, we expect subjective judgments to play a larger role in our decision making. For examples of the approach we have taken to date, see our writeup of our recent recommendation for a grant to the Innovation in Government Initiative, a grantmaking entity within the Abdul Latif Jameel Poverty Action Lab (J-PAL) or our page evaluating phase I of our 2016 grant to Results for Development (R4D). While writing about such judgments will be a challenge of this work, we are fully committed to sharing what has led us to our decisions, with only limited exceptions due to confidential or sensitive information.

What does this mean for staffing and organizational growth?

We need to grow our team to achieve our goals. Repeatedly this past year, we had to make the difficult choice to not take on a research project or investigate a grant opportunity that seemed promising because we did not have the capacity.

We are planning to roughly double our research team over the next few years, primarily by adding researchers who have experience and/or an academic background in global health and development. We are looking to add both individual contributors and research managers to the team. We expect that the people we hire in the next few years will play a critical role in shaping GiveWell’s future research agenda and will be some of the leaders of GiveWell in the future.

For more information about the research roles we’re hiring for, see our jobs.

The post How GiveWell’s research is evolving appeared first on The GiveWell Blog.

GiveWell is dedicated to finding outstanding giving opportunities and publishing the full details of our analysis. In addition to evaluations of other charities, we publish substantial evaluation of our own work. This post lays out highlights from our 2017 metrics report, which reviews what we know about how our research impacted donors. Please note:

• We report on “metrics years” that run from February through January; for example, our 2017 data cover February 1, 2017 through January 31, 2018.
• We differentiate between our traditional charity recommendations and the work of the Open Philanthropy Project, which became a separate organization in 2017 and whose work we exclude from this report.
• More context on the relationships between GiveWell, Good Ventures, and the Open Philanthropy Project can be found here.

Summary of influence: In 2017, GiveWell influenced charitable giving in several ways. The following table summarizes our understanding of this influence.

Headline money moved: In 2017, we tracked $117.5 million in money moved to our recommended charities. Our money moved only includes donations that we are confident were influenced by our recommendations.

Money moved by charity: Our nine top charities received the majority of our money moved. Our seven standout charities received a total of $1.8 million.

Money moved by size of donor: In 2017, the number of donors and amount donated increased across each donor size category, with the notable exception of donations from donors giving $1,000,000 or more. In 2017, 90% of our money moved (excluding Good Ventures) came from 20% of our donors, who gave $1,000 or more.

Donor retention: The total number of donors who gave to our recommended charities or to GiveWell unrestricted increased about 29% year-over-year to 23,049 in 2017. This included 14,653 donors who gave for the first time. Among all donors who gave in the previous year, about 42% gave again in 2017, up from about 35% who gave again in 2016.

Our retention was stronger among donors who gave larger amounts or who first gave to our recommendations prior to 2015. Of larger donors (those who gave $10,000 or more in either of the last two years), about 73% who gave in 2016 gave again in 2017.

GiveWell’s expenses: GiveWell’s total operating expenses in 2017 were $4.6 million. Our expenses decreased from about $5.5 million in 2016 due to the Open Philanthropy Project becoming a separate organization in June 2017. We estimate that 67% of our total expenses ($3.1 million) supported our traditional top charity work and about 33% supported the Open Philanthropy Project. In 2016, we estimated that expenses for our traditional top charity work were about $2.0 million.

Donations supporting GiveWell’s operations: GiveWell raised $5.7 million in unrestricted funding (which we use to support our operations) in 2017, compared to $5.6 million in 2016. Our major institutional supporters and the six largest individual donors contributed about 49% of GiveWell’s operational funding in 2017.

Web traffic: The number of unique visitors to our website remained flat in 2017 compared to 2016 (when excluding visitors driven by AdWords, Google’s online advertising product).

For more detail, see our full metrics report (PDF).

The post GiveWell’s money moved and web traffic in 2017 appeared first on The GiveWell Blog.

This is the third of three posts that form our annual review and plan for the following year. The first two posts covered GiveWell’s progress and plans on research. This post reviews and evaluates GiveWell’s progress last year in outreach and operations and sketches out some high-level goals for the current year. A separate post will look at metrics on our influence on donations in 2017. We aim to release our metrics on our influence on donations in 2017 by the end of June 2018.

Summary

Outreach: Before 2017, outreach wasn’t a major organizational priority at GiveWell (more in this 2014 blog post). In our plans for 2017, we wrote that we planned to put more emphasis on outreach, but were at the early stages of thinking through what that might involve. In the second half of 2017, we experimented with a number of different approaches to outreach (more on the results below). In 2018, we plan to increase the resources we devote to outreach primarily by hiring a Head of Growth and adding staff to improve our post-donation follow-up with donors.

Operations: In 2017, we completed the separation of GiveWell and the Open Philanthropy Project and increased our operations capacity with three new hires. In 2018, our top priorities are to hire a new Director of Operations (which we have now done), maintain our critical functions, and prepare our systems for increased growth in outreach.

Outreach 2017 review and 2018 plans

Before 2017, outreach wasn’t a major organizational priority at GiveWell (more in this 2014 blog post). In our plans for 2017, we wrote that we planned to put more emphasis on outreach, but were at the early stages of thinking through what that might involve.

We currently have one staff member, Catherine Hollander, who works on outreach full-time. Two others, Tracy Williams and Isabel Arjmand, each spend significant time on outreach. From August 2017, our Executive Director, Elie Hassenfeld, also started to allocate a significant amount of his time to outreach.

How did we do in 2017?

In 2017, we focused on experimentation. In brief, we found that:

• Advertising on podcasts has had strong results. Using the methodology described in this blog post, our best guess is that each dollar we spent on podcast advertising returned $5-14 in donations to our top charities.
• Increasing the consistency of our communication with members of the media had strong results for the time invested.
• Retaining a digital marketing consultant yielded strong results.
• Retaining a PR firm to generate media mentions did not have positive results.
• We’ve had a limited number of conversations with high net worth donors. We don’t yet have enough information to conclude whether this was a good use of time.

You can see our estimates of the five-year net present value of donations generated by each of these activities here. Overall, we spent approximately $200,000 and devoted significant staff time to this work. Our best estimate is that these efforts resulted in $2.5 million to $5.9 million in additional donations to our recommended charities.

We conclude:

• New work on outreach had a high return on investment in 2017.
• Some activities, such as podcast advertising and digital marketing improvements, have shown particularly strong results and should be scaled up.

What are our priorities for 2018?

Our marketing funnel has three stages:

1. Awareness/acquisition: more people hear about GiveWell and visit the website,
2. Conversion: more people who visit the site donate, and
3. Retention: over time, donors maintain or increase their donations.

Our current working theory is that we should prioritize (though not exclusively) improving the bottom of this funnel (retention and conversion) before moving more people through it. We also plan to scale up the activities that worked well in 2017 and to continue experimenting with different approaches.

Our primary outreach priorities (which we expect to achieve and devote substantial capacity to) for 2018 are:

1. Hire a Head of Growth to improve our efforts to acquire and convert new donors via our website. Over the long term, the Head of Growth will be responsible for digital marketing.

   What does success look like? Hire a Head of Growth.

2. Improve the post-donation experience. We believe we have substantial room to improve our post-donation communication with donors. We have hired a consultant to help us improve our process.

   What does success look like? Significantly improve our process for post-donation follow-up before giving season 2018.

   At this point, we’re still in the earliest stages of figuring out how we’ll do this, so we don’t have concrete goals for the year beyond finalizing our plan in the next few months. Our stretch goal for the year is to succeed in achieving measured improvement in our dollar retention rate/lifetime value of each donor.

Our secondary outreach priorities (which we expect to achieve, but not devote substantial capacity to) for 2018 are:

3. Continue advertising on podcasts. This advertising was particularly successful in 2017. We want to systematically assess podcast advertising opportunities and increase our podcast advertising. We plan to spend approximately $250,000 to $350,000 on podcast advertising this year.

   What does success look like? Advertise on new podcasts and measure results to decide how much to spend in 2019.

4. Receive coverage in major news outlets. This has led to increased donations in the past.

   What does success look like? Pitch major news outlets on at least five stories in total and get at least one story covered.

5. Deepen relationships with the effective altruism community. We want to deepen our relationships with groups in the effective altruism community doing outreach, particularly to high net worth donors.

For a list of other potentially promising projects we’re unlikely to prioritize this year, see this spreadsheet.

Operations 2017 review and 2018 plans

In 2017, we increased our operations staff capacity, made a number of changes to our internal systems, and completed the separation of GiveWell and the Open Philanthropy Project. In addition to maintaining critical functions, our highest priorities for 2018 are to (i) appoint a new Director of Operations and (ii) make improvements to our processes across the board to prepare our systems for major growth in outreach.

How did we do in 2017?

We made a number of improvements to our operations. In brief:

• We completed the separation of GiveWell and the Open Philanthropy Project.
• Donations: We hired two new members of our donations team, which allowed us to process donations consistently notwithstanding increased volume. We also added Betterment and Bitpay (for Bitcoin) as donation options.
• Finance: We hired a Controller. We rolled out a few systems to improve the efficiency of our internal processes (Expensify, Bill.com, and others).
• Social cohesion: We created a regular schedule for visit days for remote staff and staff events to maintain cohesion.

In January 2018, Sarah Ward, our former Director of Operations, departed. Natalie Crispin (Senior Research Analyst) has been covering her previous responsibilities during our search for a new hire to take them on.

What are our priorities for 2018?

In the first half of 2018, we aim to move from a situation in which we were maintaining critical functions to positioning the organization to grow.

Our two main priorities for the first half of 2018 are to:

1. Appoint a new Director of Operations (complete). In April 2018, we hired Whitney Shinkle as our new Director of Operations. Between January and April 2018, Natalie Crispin served as our interim Director of Operations.
2. Prepare our systems for major growth in outreach, which we expect to lead to increases in spending, staff, and donations.
3. Maintain critical operations across domains: donations, finance, HR, office, website, recruiting, and staff cohesion.

Major operations projects we aim to complete in the first half of 2018 include:

• A significant improvement in our approach to budgeting making it significantly easier for us to share updated actual spending versus budget.
• We retained a compensation consultant to help us benchmark GiveWell staff compensation to comparable organizations.
• We published our 2016 metrics report and plan to publish our 2017 money moved report by the end of June.

The post GiveWell’s outreach and operations: 2017 review and 2018 plans appeared first on The GiveWell Blog.

This is the first of three posts that form our annual review and plan for the following year. This post reviews and evaluates last year’s progress on our work of finding and recommending evidence-based, thoroughly-vetted charities that serve the global poor. The following two posts will cover (i) our plans for GiveWell’s research in 2018 and (ii) GiveWell’s progress and plans as an organization. We aim to release our metrics on our influence on donations in 2017 by the end of June 2018.

Summary

We believe that 2017 was a successful year for GiveWell’s research. We met our five primary goals for the year, as articulated in our plan post from the beginning of the year:

Our primary research goals for 2017 are to:

1. Speed up our output of new intervention assessments, by hiring a Senior Fellow and by improving our process for reviewing interventions at a shallow level.
2. Increase the number of promising charities that apply for our recommendation. Alternatively, we may learn why we have relatively few strong applicants and decide whether to change our process as a result. Research Analyst Chelsea Tabart will spend most of her time on this project.
3. Through GiveWell Incubation Grants, fund projects that may lead to more top charity contenders in the future and consider grantees No Lean Season and Zusha! as potential 2017 top charities.
4. Further improve the robustness and usability of our cost-effectiveness model.
5. Improve our process for following the progress of current top charities to reduce staff time, while maintaining quality. We also have some specific goals (discussed below) with respect to answering open questions about current top charities.

We achieved our five primary goals for the year:

1. Our intervention-related output was greater than in any past year, although we still see room for improvement in the pace with which we complete and publish this work (more). We hired a Senior Fellow and published nine full or interim intervention reports in 2017, compared to four in 2016.
2. We increased the number of promising charities that applied for our recommendation (more).
3. We added two new top charities: Evidence Action’s No Lean Season (the first top charity to start as a GiveWell Incubation Grant recipient) and Helen Keller International’s vitamin A supplementation program (which joined our list as a result of our charity outreach work). We continued to follow our current Incubation Grant recipients and made several new Incubation Grants to grow the pipeline of new top charities (more).
4. We made substantial improvements to our cost-effectiveness analysis (more).
5. We reduced the amount of staff time spent on following our current top charities. We also completed 17 of the 19 activities outlined in last year’s plan (more).

We discuss progress on each of our primary goals below. For each high-level goal, we include (i) the subgoals we set in our last annual review, (ii) an evaluation of whether we met those subgoals, and (iii) a summary of key activities completed last year.

Goal 1: Speed up intervention assessments

In early 2017, we wrote:

In recent years, we have completed few intervention reports, which has limited our ability to consider new potential top charities. We plan to increase the rate at which we form views on interventions this year by:

• Hiring a Senior Fellow (or possibly more than one). We expect a Senior Fellow to have a Ph.D. in economics, public health, or statistics or equivalent experience and to focus on in-depth evidence reviews and cost-effectiveness assessments of interventions that appear promising after a shallower investigation. In addition, Open Philanthropy Project Senior Advisor David Roodman may spend some more time on intervention related work.
• Doing low-intensity research on a large number of promising interventions. We generally start with a two to four hour “quick intervention assessment,” and then prioritize interventions for a 20-30 hour “interim intervention report” (example). We don’t yet have a good sense of how many of these of these we will complete this year, because we’re unsure both about how much capacity we will have for this work and about how many promising interventions there will be at each step in the process.
• Continuing to improve our systems for ensuring that we become aware of promising interventions and new relevant research as it becomes available. We expect to learn about additional interventions by tracking new research, particularly randomized controlled trials, in global health and development and by talking to select organizations about programs they run that they think we should look into.

How did we do? Achieved our goal.

Due to our uncertainty about the capacity we could devote to intervention assessments, we did not have an explicit target for how many reports we expected to complete. In 2017, we published seven interim intervention reports, two full intervention reports, and completed ~30 quick evidence assessments (defined below). Our research output for 2017 was higher than 2016, when we published one full intervention report, three interim intervention reports, and completed 30 quick evidence assessments.

What did we do?

• Hired Caitlin McGugan as a full-time Senior Fellow to work on intervention reports.
• Completed quick evidence assessments (two to four hours spent reviewing the evidence base) for ~30 programs. (We plan to write more about this process, what we found, and how it informs our work in a future blog post.)
• Published two full intervention reports: (i) conditional subsidies for seasonal labor migration in northern Bangladesh (which led to a new top charity, No Lean Season) and (ii) surgery to repair obstetric fistula.
• Completed seven interim intervention reports, which are less intensive than full intervention reports, on antiretroviral therapy to treat HIV/AIDS, oral rehydration solution, Sayana® Press, SMS reminders for vaccination, distribution of eyeglasses in developing countries, interventions to promote handwashing, and oral pre-exposure prophylaxis (PrEP) for HIV.
• Substantially updated intervention reports on two of our priority programs: seasonal malaria chemoprevention (previously an interim intervention report) and vitamin A supplementation.
• Developed and implemented a system for (i) tracking new published research relevant to our priority programs and (ii) identifying new programs for evidence assessments.

Goal 2: Increase the pipeline of promising charities applying for our recommendation

In early 2017, we wrote:

We would like to better understand whether we have failed to get the word out about the potential value we offer or communicate well about our process and charities’ likelihood of success, or, alternatively, whether charities are making well-informed decisions about their fit with our criteria. (More on why we think more charities should consider applying for a GiveWell recommendation in this post.)

This year, we have designated GiveWell Research Analyst Chelsea Tabart as charity liaison. Her role is to increase and improve our pipeline of top charity contenders by answering charities’ questions about our process and which program(s) they should apply with, encouraging promising organizations to apply, and, through these conversations, understanding what the barriers are to more charities applying.

We aim by the end of the year to have a stronger pipeline of charities applying, have confidence that we are not missing strong contenders, or understand how we should adjust our process in the future.

How did we do? Achieved our goal.

More charities entered our top charity review process in 2017, although it’s unclear whether this was due to our charity liaison activities. Five charities formally applied in 2017, compared to two in 2016, and four in 2015. One of those charities, Helen Keller International’s vitamin A supplementation program, became a top charity.

While we feel our relationships with well-regarded global health and development implementers and funders have improved, we continue to feel limited in our ability to understand whether there are funding gaps for evidence-backed, highly cost-effective work within large international NGOs and multilateral aid organizations such as the Global Fund to Fight AIDS, Tuberculosis and Malaria.

What did we do?

• We had at least one conversation with 42 organizations to introduce them to GiveWell’s work in 2017, compared to 16 in 2016.
• Where organizations running multiple programs expressed interest in applying for our recommendation, we had several calls with them to help determine whether they should apply and which of their programs would be the most promising fit for a top charity evaluation. We had not offered this proactive support to organizations in the past.
• We hosted two charity-focused events: (i) a conference call for charities with GiveWell senior staff to present an update on our work as it relates to charities and to give them a chance to ask questions directly of our senior team and (ii) a networking event for our recommended organizations in London.
• We attended seven conferences on global health and development issues to broaden our network and perspective in subject-matter areas that GiveWell has not historically worked on.

Goal 3: Maintain Incubation Grants

In early 2017, we wrote:

We made significant progress on Incubation Grants in 2016 and plan in 2017 to largely continue with ongoing engagements, while being open to new grantmaking opportunities that are brought to our attention.

Among early-to-mid stage grants, we plan to spend the most time on working with IDinsight and New Incentives (where our feedback is needed to move the projects forward), and a smaller amount of time on Results for Development and Charity Science: Health (where we are only following along with ongoing projects).

Another major priority will be following up on two later-stage grantees, No Lean Season and Zusha!, groups that are contenders for a top charity recommendation in 2017. For No Lean Season, a program run by Evidence Action, our main outstanding questions are whether the program will have room for more funding in 2018 and whether monitoring will be high quality as the program scales. We have similar questions about Zusha! and in addition are awaiting randomized controlled trial results that are expected later this year.

How did we do? Exceeded goal.

As expected, our work last year focused on following up on current grantees. No Lean Season, one of our later-stage grantees, graduated to top charity status and we made one grant to a new grantee, the Centre for Pesticide Suicide Prevention. We also made a number of grants to improve our understanding of the evidence base for our priority programs and deepened our partnership with IDinsight.

What did we do?

• Completed a full review of No Lean Season and named it a top charity. We are currently reviewing Zusha! in greater depth.
• Made Incubation Grants totaling $10.8 million in 2017:
  • $7.8 million to support the continued operation of previous grantees: Evidence Action, Zusha!, New Incentives, Charity Science Health, and the IDinsight embedded GiveWell team.
  • $3.0 million to new Incubation Grant recipients: the Centre for Pesticide Suicide Prevention, UC Berkeley for a long-run follow-up survey on the effect of deworming on consumption, and the Center for Effective Global Action at UC Berkeley to scope other randomized controlled trials (RCTs) for long-term follow-ups.

Goal 4: Improve our cost-effectiveness analysis

In early 2017, we wrote:

We plan to continue making improvements to our cost-effectiveness model and the data it draws on (separate from adding new interventions to the model, which is part of the intervention report work discussed above). Projects we are currently prioritizing include:

• Making it more straightforward to see how personal values are incorporated into the model and what the implications of those values are.
• Revisiting the prevalence and intensity adjustment that we use to compare the average per-person impact of deworming in places that our top charities work to the locations where the studies that found long-term impact of deworming were conducted. More in this post.
• Improving the insecticide-treated nets model by revisiting how it incorporates effects on adult mortality and adjustments for regions with different malaria burdens and changes in malaria burden over time.

How did we do? Achieved goal.

We made substantial progress on improving our cost-effectiveness analysis in 2017.

What did we do?

• Moved to a system of making more frequent updates to our cost-effectiveness analysis. This has made it easier to identify which specific factors are driving changes in the estimated cost-effectiveness of our top charities.
• Revisited how we think about leverage and funging (how donating to our top charities influences how other funders spend their money) and updated our cost-effectiveness analysis accordingly.
• Published a report on how other global actors approach the difficult moral tradeoffs we face.
• Prior to announcing our 2017 recommendations, we performed a sensitivity check on our cost-effectiveness analysis to identify how sensitive our final outputs were to different uncertain inputs. This has helped us identify which inputs we should prioritize additional research on, and we believe it has made our communication more transparent, particularly around our personal values.
• Revisited and updated our prevalence and intensity adjustments for deworming.
• Deprioritized improving how our insecticide-treated net model incorporates effects on adult mortality. A limited number of conversations with malaria experts made us less confident that there was informative research on the question that would improve the accuracy of our models.
• Deprioritized making adjustments for subnational regions with different malaria burdens because it would take substantial time to deeply understand the assumptions informing the subnational models we have seen. We believe this remains an important weakness of our model and that it limits our ability to make high-quality decisions about prioritization among different regional funding gaps.

Goal 5: Improve our process for following top charities

In early 2017, we wrote:

“In 2017, we plan to have a single staff member do most of this work and expect it to take a half to two-thirds of a full-time job. Three other staff will spend a small portion of their time, totaling approximately the equivalent of one full-time job, on this work.”

How did we do? Achieved goal.

We estimate that it took about 40 percent of the staff member’s time who focused on this work plus a small portion of four other staff members’ time, totaling at most and likely somewhat less than the equivalent of a full-time job (roughly half the time we dedicated to top charity updates in 2016).

We believe we maintained or increased the quality of the top charity updates, as we completed or made major progress on all but two of the activities and questions outlined in last year’s plan.

What did we do?

The table below summarizes our progress on each of the activities and open questions outlined in last year’s plan.

Charity Goals and open questions from 2017 plan Did we meet our goal? What did we do? Evidence Action’s Deworm the World Initiative “We have now followed these groups for several years and do not have major outstanding questions about them. We plan to ask for updates on financial information, monitoring results, and room for more funding and have regular phone calls with them to learn about operational changes that might lead us to ask additional questions.” Yes We had regular phone calls, received up-to-date financial information, updated room for more funding, and reviewed new monitoring information from Nigeria, Vietnam, Kenya, and Ethiopia (see rows 11-20 and an overview of what we learned). GiveDirectly “We have now followed these groups for several years and do not have major outstanding questions about them. We plan to ask for updates on financial information, monitoring results, and room for more funding and have regular phone calls with them to learn about operational changes that might lead us to ask additional questions.” Yes We had regular phone calls, received up-to-date financial information, updated room for more funding, and reviewed new monitoring information from Kenya (1, 2). (Overview of what we learned.) Schistosomiasis Control Initiative “We have now followed these groups for several years and do not have major outstanding questions about them. We plan to ask for updates on financial information, monitoring results, and room for more funding and have regular phone calls with them to learn about operational changes that might lead us to ask additional questions.” Yes We had regular phone calls, received up-to-date financial information, updated room for more funding, and reviewed new monitoring information from 2016 programs in a number of countries. (Monitoring information, overview of what we learned.) Against Malaria Foundation (AMF) “Will AMF’s monitoring processes be high quality?” Yes We commissioned IDinsight, an organization with which we are partnering as part of our Incubation Grants program, to observe post-distribution surveys in Malawi and Ghana and report their findings. “Going forward, AMF aims to fund larger distributions and commit funding further ahead of when a distribution is scheduled to occur than it has, for the most part, done in the past. Will this increase the extent to which AMF funds displace funds from other sources, or will there continue to be evidence that AMF’s funds are largely adding to the total number of nets distributed?” Partial We learned relatively little about the displacement/fungibility question because AMF signed relatively few new agreements to fund long-lasting insecticide-treated net distributions in 2017. There was an update to how AMF will be tracking displacement, described in the second paragraph here. “In order to estimate AMF’s room for more funding, we will seek out information on the location and size of funding gaps for mass net distribution campaigns from AMF, the African Leaders Malaria Alliance, and possibly other funders of nets. As we have in the past, we will use this information in conjunction with conversations with AMF about non-funding bottlenecks to its ability to fill various gaps.” Yes We got updates on AMF’s room for more funding, as summarized in this post. The END Fund’s deworming program “We have not yet seen monitoring on par with that from our other top charities from the END Fund. We expect results from coverage surveys from END Fund programs this year. Will these surveys be high quality and demonstrate that the END Fund is funding successful programs?” Yes We saw some monitoring from END Fund programs; previously our recommendation of the END Fund was based on specific monitoring plans that we found credible (more here). “We have not yet tried to compare the cost-effectiveness of the END Fund to our other top charities in our cost-effectiveness model. We will be seeking additional information from the END Fund about cost per treatment and baseline infection rates” Yes We significantly improved our understanding of the END Fund’s cost per treatment and the baseline prevalence in areas where the END Fund works. We completed a cost-effectiveness analysis, though we continue to have lower confidence in our estimates than we do for the deworming organizations that we have recommended for several years. “Questions around room for more funding: the extent to which funding due to GiveWell’s recommendation increases the amount that the END Fund spends on deworming versus other programs, actual and projected revenue from other sources, and what deworming grantmaking opportunities the END Fund expects to have.” Yes We estimated the extent to which funding due to GiveWell’s recommendation increases the amount that the END Fund spends on deworming versus other programs, discussed here. “We visited the END Fund’s programs in Rwanda and Idjwi island, DRC in January 2017 and will publish notes and photos from our visit shortly.” Yes We posted notes and photos from our site visit here. Malaria Consortium’s seasonal malaria chemoprevention program “Further research on the evidence of effectiveness, cost-effectiveness, and potential downsides of seasonal malaria chemoprevention (SMC) (due to time constraints we have not yet completed a full intervention report, though we felt sufficiently confident in the intervention to recommend Malaria Consortium).” Yes We reviewed each of the RCTs included in the Cochrane review for seasonal malaria chemoprevention, and possible negative/offsetting impacts. We updated our interim intervention report to a full intervention report and added new information to our cost-effectiveness analysis. Our key conclusions did not change substantially and SMC remains a priority program. “Getting a better understanding of the methodology Malaria Consortium uses for estimating coverage rates.” Yes We spoke with Malaria Consortium to understand how they measure coverage and updated our cost-effectiveness analysis to account for different levels of coverage in the Malaria Consortium program relative to the headline results of the RCTs in the Cochrane review (conversation notes here). “Completing a more in-depth room for more funding analysis for the program for 2018 than we did for 2017.” Yes We completed a significantly more in-depth room for more funding analysis than we had previously (more here). “We may visit a Malaria Consortium seasonal malaria chemoprevention program in summer 2017.” No We did not conduct a site visit. Sightsavers’ deworming program “We expect to make limited progress this year because the first deworming mass drug administration funded with GiveWell-influenced funds is not expected to take place until September at the earliest and monitoring results aren’t expected until early 2018. Because Sightsavers has done fairly little deworming in the past year, we don’t expect to be able to learn much from its ongoing programs.” Exceeded In 2017, as expected, we learned relatively little about the performance of Sightsavers’ deworming programs, because programs funded with GiveWell-directed funds were at early stages. We did not expect to receive any monitoring results from programs funded with GiveWell-directed funds; however, Sightsavers shared a coverage survey from Guinea with us earlier than expected. The survey found middling coverage results. “Getting more information from Sightsavers about baseline prevalence and intensity of worm infections in the areas it is working, to inform our cost-effectiveness analysis.” Yes We significantly improved our understanding of Sightsavers’ cost per treatment and the baseline prevalence in areas where Sightsavers works (which is used in our cost-effectiveness analysis). “Using Sightsavers’ budget for the projects and planned treatment numbers to improve our estimate of the cost per treatment – another input into our cost-effectiveness analysis.” Yes We significantly improved our understanding of Sightsavers’ cost per treatment and the baseline prevalence in areas where Sightsavers works (which is used in our cost-effectiveness analysis). “Completing a room for more funding analysis for 2018.” Yes We completed a room for more funding analysis (more here). Standout charities “We plan to have at least one phone call with each of these groups to discuss whether anything has changed that might lead us to reopen consideration of the organization as a potential top charity” Yes We spoke with each standout charity. Conversation notes here: Development Media International, Food Fortification Initiative, Global Alliance for Improved Nutrition’s Universal Salt Iodizational program, Iodine Global Network, Living Goods, and Project Healthy Children.

The post Review of our research in 2017 appeared first on The GiveWell Blog.

In September 2017, we posted an interim update on GiveWell’s 2016 money moved and web traffic. This post summarizes the key takeaways from our full 2016 money moved and web traffic metrics report. Note that some of the numbers, including the total headline money moved, have changed since our interim report. Since then, we decided to exclude some donations from our headline money moved figure (details in the full report), and we corrected some minor errors.

This report was highly delayed (as discussed in the interim update). We expect to publish our report on GiveWell’s 2017 money moved and web traffic much more quickly; our current expectation is that we will publish that report by the end of June.

GiveWell is dedicated to finding outstanding giving opportunities and publishing the full details of our analysis. In addition to evaluations of other charities, we publish substantial evaluation of our own work. This post lays out highlights from our 2016 metrics report, which reviews what we know about how our research impacted donors. Please note:

• We report on “metrics years” that run from February through January; for example, our 2016 data cover February 1, 2016 through January 31, 2017.
• We differentiate between our traditional charity recommendations and our work on the Open Philanthropy Project, which became a separate organization in 2017 and whose work we exclude from this report.
• More context on the relationship between Good Ventures and GiveWell can be found here.

Summary of influence: In 2016, GiveWell influenced charitable giving in several ways. The following table summarizes our understanding of this influence.

Headline money moved: In 2016, we tracked $88.6 million in money moved to our recommended charities. Our money moved only includes donations that we are confident were influenced by our recommendations.

Money moved by charity: Our seven top charities received the majority of our money moved. Our six standout charities received a total of $2.9 million.

Money moved by size of donor: In 2016, the number of donors and amount donated increased across each donor size category, with the notable exception of donations from donors giving $1,000,000 or more. In 2016, 93% of our money moved (excluding Good Ventures) came from 19% of our donors, who gave $1,000 or more.

Donor retention: The total number of donors who gave to our recommended charities or to GiveWell unrestricted increased about 16% year-over-year to 17,834 in 2016. This included 12,461 donors who gave for the first time. Among all donors who gave in the previous year, about 35% gave again in 2016, down from about 40% who gave again in 2015.

Our retention was stronger among donors who gave larger amounts or who first gave to our recommendations prior to 2014. Of larger donors (those who gave $10,000 or more in either of the last two years), about 77% who gave in 2015 gave again in 2016.

GiveWell’s expenses: GiveWell’s total operating expenses in 2016 were $5.5 million. Our expenses increased from about $3.4 million in 2015 as the size of our staff grew and average seniority level rose. We estimate that about one-third of our total expenses ($2.0 million) supported our traditional top charity work and about two-thirds supported the Open Philanthropy Project. In 2015, we estimated that expenses for our traditional charity work were about $1.1 million.

Donations supporting GiveWell’s operations: GiveWell raised $5.6 million in unrestricted funding (which we use to support our operations) in 2016, compared to $5.1 million in 2015. Our major institutional supporters and the five largest individual donors contributed about 70% of GiveWell’s operational funding in 2016. This is driven in large part by the fact that Good Ventures funded two-thirds of the costs of the Open Philanthropy project, in addition to funding 20% of GiveWell’s other costs.

Web traffic: The number of unique visitors to our website was down very slightly (by 1%) in 2016 compared to 2015 (when excluding visitors driven by AdWords, Google’s online advertising product).

For more detail, see our full metrics report (PDF).

The post GiveWell’s money moved and web traffic in 2016 appeared first on The GiveWell Blog.

GiveWell’s impact is a function of the quality of our research and the amount of money we direct to our recommended charities (our “money moved”). Historically, we’ve focused mostly on research because we felt that the quality of our recommendations was a greater constraint to our impact than our money moved.

This has changed. Outreach is now a major organizational priority. The goal of this work is to increase the amount of money we direct to our top-recommended charities.

In April 2014 I wrote about our work on outreach to explain why we hadn’t prioritized it: in brief, our growth had largely been driven by inbound interest in GiveWell, and proactive outreach efforts (beyond building relationships with existing donors) hadn’t yielded results that were worth the cost.

What changed?

• We believe that the amount of money we move is now a greater constraint to our impact than additional improvements in the quality of our research. Over the last two years, we’ve added five new top charities (three of which implement programs that weren’t previously represented on our top charities list), and we expect that our top charities, collectively, will have more than $200 million in unfilled funding gaps once they’ve received the funding that we expect to direct to them. (This calculation excludes GiveDirectly, which we believe could absorb and distribute hundreds of millions of dollars.) At the same time, the quality of our research and our capacity for research is higher than it’s ever been, so the returns to adding staff there (in terms of the pace at which we identify significantly better giving opportunities) are now lower.
• Increased capacity for outreach. In our 2014 post, we wrote that one of our key constraints was that senior staff (which at the time meant primarily GiveWell Co-Founder Holden Karnofsky and me) were necessary for most outreach-related work. This has changed. We now have capacity to take on outreach work as other staff have been hired and trained on this type of work.
• Better information on the impact of GiveWell’s outreach. We now have better information about the returns to outreach because:
  1. We’ve collected better data (via an improved donations processing system and outreach efforts) about where donors find out about us. Because of our ability to track donors, we know that a single appearance on NPR or major podcasts tends to drive $50,000+ in annual donations.
  2. More time passing has demonstrated that the lifetime value of the donations of a first time donor is higher than we expected. In several cases, we’ve seen major donors (i.e., those giving $10,000-$100,000) increase their annual giving by a factor of 10 or more.

We’re in the early stages of figuring out how we can proactively invest time and money in outreach to significantly increase our money moved. For now, we’ve taken some opportunities that we think will have positive returns; these are the three that we’ve invested the most time and money in to date:

• Podcast advertising. We’ve been advertising on podcasts that we believe our target audience listens to, based on interviews with current donors and GiveWell staff. In February and March, we ran a small experiment with a few ads on FiveThirtyEight’s Politics podcast and Vox’s The Weeds.1We’ve also been running ads on Julia Galef’s Rationally Speaking podcast since then. Because it’s much smaller and more targeted, we’ve excluded it from this analysis. Measured returns to advertising on Rationally Speaking have been significantly better than the more mainstream podcasts discussed in this post. jQuery("#footnote_plugin_tooltip_1").tooltip({ tip: "#footnote_plugin_tooltip_text_1", tipClass: "footnote_tooltip", effect: "fade", fadeOutSpeed: 100, predelay: 400, position: "top right", relative: true, offset: [10, 10] });

  In total, we spent approximately $20,000 on ads for this initial experiment. We ask donors who give via our website to tell us where they learned about GiveWell when they donate. GiveWell received approximately $8,000 in donations between February 1 and November 20 from donors who reported that they had learned about us via these podcasts.

  The donations we received were from first-time donors; to assess the impact of our advertising, we need to estimate the lifetime value of acquiring a new donor. In work we’ve done to assess our retention rate, we’ve seen that (a) approximately 20-25% of the donors who make a first-time donation of less than $1,000 give again in the subsequent year but (b) because many first-time donors increase the size of their donation over time, collectively, the donors who recur give more than 100% of the value of what they give in their first year.

  At higher donation levels ($1,000-$100,000), we measure 40-45% retention among donors, which leads to retention of approximately two-thirds of dollars given.2I say “measure” retention because we’ve learned that many donors give subsequent donations directly to our top charities and don’t report those donations to us. We’ve tried to follow up with lapsed donors and with charities to track these donors down. jQuery("#footnote_plugin_tooltip_2").tooltip({ tip: "#footnote_plugin_tooltip_text_2", tipClass: "footnote_tooltip", effect: "fade", fadeOutSpeed: 100, predelay: 400, position: "top right", relative: true, offset: [10, 10] });

  We therefore estimated the net present value of expected future donations (over the next five years) from these podcasts ads as somewhere between approximately $20,000 (assuming two-thirds dollar retention for the first two years and 100% dollar retention subsequently) and $45,000 (assuming 100% dollar retention).3We only projected donations over five years. This is fairly arbitrary because we don’t have long-term enough data to know whether or not this is a reasonable assumption. We capped it to prevent our assessment being driven by speculation about how much money would be donated many years in the future. jQuery("#footnote_plugin_tooltip_3").tooltip({ tip: "#footnote_plugin_tooltip_text_3", tipClass: "footnote_tooltip", effect: "fade", fadeOutSpeed: 100, predelay: 400, position: "top right", relative: true, offset: [10, 10] });

  A few additional facts are worth keeping in mind about the above figures:

  • We ran this experiment in February and March; most donors give at the end of the calendar year. We consistently see donors who find out about GiveWell during the course of the year, but donate in December. Other things equal, we expect that our advertising would have had greater measured returns in December than earlier in the year.
  • We are only able to track donors who (a) fill out our donation form telling us where they learned about us and (b) give directly through our website rather than to our top charities. Less than 50% of donors who give via credit card (and a smaller percentage of donors who give via check) tell us where they learned about GiveWell. Also, roughly speaking, approximately 50% of the donors and dollars we influence come through GiveWell rather than going to our top charities.4I took this rough estimate from footnote 26, on page 15, of GiveWell’s 2015 metrics report. jQuery("#footnote_plugin_tooltip_4").tooltip({ tip: "#footnote_plugin_tooltip_text_4", tipClass: "footnote_tooltip", effect: "fade", fadeOutSpeed: 100, predelay: 400, position: "top right", relative: true, offset: [10, 10] });
  • It’s certainly possible that donors who learn about us via podcast would be more likely to give through our website than an average donor, more likely to report on how they found us (since their source is clear), or less likely to be retained. My best guess is that donors who learn about us via podcast ads behave similarly to our other donors, but I won’t be surprised if they don’t.

  With all that in mind, I believe that the impact of our podcast advertising is higher than what we directly measured.

  The results we saw from February to November this year were promising enough that we decided to increase the size of our experiment by spending approximately $100,000 on podcast ads. We’re currently running ads on FiveThirtyEight’s Politics podcast and Ezra Klein’s podcast and The Weeds at Vox.

• Earned media outreach. Mentions of GiveWell in the media have historically been a strong driver of growth. We aimed to increase mentions of GiveWell in high-quality, high-profile media where we’ve had the most past success as measured by dollars donated (i.e., media like The New York Times, NPR, The Wall Street Journal, and Financial Times). We retained a PR firm that came strongly recommended; we also increased 1-to-1 outreach by GiveWell staff to members of the media who have covered GiveWell in the past. It’s very hard to attribute the impact of the additional effort we’ve invested—overall, our effort has been fairly limited, and it’s hard to easily draw the causal lines between our work and the stories that appear—but my guess is that our increased efforts have led to more coverage of GiveWell and our top charities this giving season than in the recent past.
• Website improvements. Companies that sell products online invest significant effort into optimizing their websites and checkout pages to maximize their revenues. We retained a marketing consultant, Will Wong of Mission Street, and we’ve been A/B testing different donation pages and plan to test other pages on our website such as our homepage or top charities page to see whether we can increase our conversion rate (i.e., the percentage of visitors to our website who give to one of our top charities). For context, our current conversion rate is 1%. Our understanding is that a standard conversion rate for e-commerce companies is 2%, and that international nonprofits have a similar conversion rate.5See Pg 51 of the study downloadable here. jQuery("#footnote_plugin_tooltip_5").tooltip({ tip: "#footnote_plugin_tooltip_text_5", tipClass: "footnote_tooltip", effect: "fade", fadeOutSpeed: 100, predelay: 400, position: "top right", relative: true, offset: [10, 10] }); An increase in our conversion rate to the industry average would lead to a significant increase in the amount of money we direct to our top charities.

Notes   [ + ]

1. ↑ We’ve also been running ads on Julia Galef’s Rationally Speaking podcast since then. Because it’s much smaller and more targeted, we’ve excluded it from this analysis. Measured returns to advertising on Rationally Speaking have been significantly better than the more mainstream podcasts discussed in this post. 2. ↑ I say “measure” retention because we’ve learned that many donors give subsequent donations directly to our top charities and don’t report those donations to us. We’ve tried to follow up with lapsed donors and with charities to track these donors down. 3. ↑ We only projected donations over five years. This is fairly arbitrary because we don’t have long-term enough data to know whether or not this is a reasonable assumption. We capped it to prevent our assessment being driven by speculation about how much money would be donated many years in the future. 4. ↑ I took this rough estimate from footnote 26, on page 15, of GiveWell’s 2015 metrics report. 5. ↑ See Pg 51 of the study downloadable here. function footnote_expand_reference_container() { jQuery("#footnote_references_container").show(); jQuery("#footnote_reference_container_collapse_button").text("-"); } function footnote_collapse_reference_container() { jQuery("#footnote_references_container").hide(); jQuery("#footnote_reference_container_collapse_button").text("+"); } function footnote_expand_collapse_reference_container() { if (jQuery("#footnote_references_container").is(":hidden")) { footnote_expand_reference_container(); } else { footnote_collapse_reference_container(); } } function footnote_moveToAnchor(p_str_TargetID) { footnote_expand_reference_container(); var l_obj_Target = jQuery("#" + p_str_TargetID); if(l_obj_Target.length) { jQuery('html, body').animate({ scrollTop: l_obj_Target.offset().top - window.innerHeight/2 }, 1000); } }

The post Update on our work on outreach appeared first on The GiveWell Blog.

In 2016, we tracked a total of $91.6 million given to our top charities as a direct result of our research.

In addition to this $91.6 million, we also directed $13.3 million to our Incubation Grants program.

A note about this report:

We have yet to complete all of the work necessary to publish our 2016 metrics report. There are a number of reasons the report has been difficult to produce this year. In our view the main factors responsible for this delay were the increasing number and complexity of the data sources used to track donors giving to our recommended charities and competing priorities that required the attention of the staff member who produced the 2016 report. The delay was not a result of a decreased dedication to transparency.

In mid-September, we committed to publishing an update on our key metrics (money moved and web traffic) by October 1st even if we were unable to complete our report by then. We have failed to complete our full report, and, today, are publishing an interim update consistent with our commitment.

Unfortunately, we don’t have an updated estimate about when we’ll publish our full metrics report. It is possible that it will take us several months or more to complete it. We view this as a major failing on our part, and we plan to correct this in the future.

For the purpose of this report, please note:

• We report on “metrics years” that run from February through January; for example, our 2016 data cover February 1, 2016 through January 31, 2017.
• We differentiate between our traditional charity recommendations, the work of the Open Philanthropy Project, and our work aiming to support the development of future GiveWell top charities. GiveWell and Open Philanthropy are now separate legal organizations, but during 2016 Open Philanthropy was part of GiveWell, so we report its grantmaking here.
• More context on the relationship between Good Ventures, Open Philanthropy, and GiveWell can be found here.

Summary of influence:

In 2016, GiveWell influenced charitable giving in several ways. The following table includes (a) donations from donors who cited our research when donating to a third party (or cited a source that recommended our top charities because of our recommendation), (b) donations to GiveWell that we granted to top charities and standout organizations, and (c) grants made on our recommendation, through GiveWell Incubation Grants and the Open Philanthropy Project.

Total money moved:

In 2016, GiveWell tracked $91.6 million in money moved to our recommended charities. Our money moved figure only includes donations that we are confident were influenced by our recommendations. In our full metrics report, we plan to include our best guess of the total funding that was given to our top and standout charities due to our research; the numbers in this blog post include only donations that we could specifically track as being due to our research. The methodology used to generate the numbers in this post was very similar to what is described in our 2015 metrics report, with the exception that, for 2016, we counted more donations through partner organizations such as Giving What We Can. We now believe that we should have included those donations last year.

Open Philanthropy gave an additional $13.3 million to the GiveWell Incubation Grants program, to support the development of future top charities, and GiveWell granted $400,000 in participation grants to organizations that applied for a top charity recommendation, from funding provided by Good Ventures for this purpose. We do not count either of these grant types in our headline money moved figure.

Money moved by charity:

Our seven top charities received the majority of our money moved. Our six standout charities received a total of about $3.1 million.

The post Interim update on GiveWell’s money moved and web traffic in 2016 appeared first on The GiveWell Blog.

This post will provide a brief overview of GiveWell’s progress in a number of areas so far this year. In summary,

1. Research: We are making progress on reaching charities that might be a good fit for a GiveWell recommendation and asking them to apply. We are also moving forward with GiveWell’s intervention prioritization goals.
2. Operations: The separation of GiveWell and the Open Philanthropy Project was a major organizational priority in the first half of the year and was finalized on June 1. We’ve also increased the specialization on the operations team and outsourced some of GiveWell’s operations work.
3. Outreach: Outreach is now a major organizational priority. We hope to develop a strategy for significantly increasing money moved to our recommended charities by September.

We hope you will let us know if you have any questions about our work this year in the comments.

Research

Our work on research now falls into two primary categories: Traditional top charities work, consisting of research into promising programs and evaluations of charities implementing them as potential GiveWell top charities, and GiveWell Incubation Grants, our work to grow the pipeline of potential top charities and improve our understanding of our current recommended charities.

Top charities

• Encouraging charities to apply for a GiveWell recommendation. In recent years, we were surprised by how few charities reached out to GiveWell to apply for a recommendation. We guessed that some part of this may be driven by (a) a lack of understanding of GiveWell’s research priorities and which organizations we might be interested in recommending, and (b) a lack of understanding or misconceptions about GiveWell’s charity review process or the value added of a GiveWell recommendation in increasing a charity’s funding.

  We took two steps to address this problem in 2017. First, GiveWell Research Analyst Chelsea Tabart is now serving as GiveWell’s “charity liaison.” In this role, Chelsea connects with groups that may be a good fit for a GiveWell recommendation to learn more about their work and to encourage them to apply if a fit seems promising, and to explain GiveWell’s review process and value added. Second, we published a blog post on why we think more charities should consider applying for a GiveWell recommendation. We are now considering a number of charities as potential top-charity contenders and attribute this in part to taking the steps described above.

• Intervention prioritization. A major goal this year is to assess a large number of interventions as potential GiveWell priority programs. This “intervention prioritization” work involves surveying the literature for a variety of interventions to identify the most cost-effective and evidence-backed programs.

  We have made good progress on intervention research in 2017. (We plan to write about our progress in more detail in a future post.) We completed 50 quick evidence assessments in the first half of the year and published interim intervention reports on:

  • Sayana Press
  • Distribution of eyeglasses in developing countries
  • Antiretroviral therapy to treat HIV/AIDS
  • Interventions to promote handwashing
  • SMS reminders for vaccination.

  We also published an intervention report on surgery to repair obstetric fistula.

GiveWell Incubation Grants

We continue to expand our work on GiveWell’s Incubation Grants program:

• Partnership with IDinsight. We continued our partnership with IDinsight. IDinsight is working on a number of projects with GiveWell, including cost-effectiveness and monitoring assessments of surgery charities (cataract and fistula), running a randomized controlled trial of New Incentives’ pilot program to provide incentives for immunizations, and helping one of our top charities, the Against Malaria Foundation, improve its monitoring.
• Considering new top charities via GiveWell Incubation Grants. We’re considering some GiveWell Incubation Grant recipient charities as potential 2017 top charities. We’ve published blog posts on Zusha! and No Lean Season as contenders for a recommendation at the end of the year.

Recruiting

We hired Caitlin McGugan as a Senior Fellow and James Snowden started working with us as a research consultant. We hope they will increase GiveWell’s output of intervention reports. We also have one summer research analyst, Scott Weathers, working with us.

Operations

The separation of GiveWell and the Open Philanthropy Project was a major priority for GiveWell’s operations team in the first half of 2017. The separation was finalized on June 1.

The operations team continues to increase in specialization (historically, GiveWell operations work has been done by generalists on staff). We hired Maryana Pinchuk to serve as a Donations Manager and Erin Wolff as a Donations Relations Assistant; a search for a Controller to manage our finance and accounting is underway. We have also started to work with new vendors to outsource some operations work, which we hope will increase our available staff capacity and improve the quality of our operations.

Outreach

Outreach is now a major organizational priority for GiveWell. In the past, we focused very little on efforts to reach new potential donors with GiveWell’s work. Now, we think outreach is more of a limiting factor than research—the high-value funding gaps we’ve identified exceed the amount of donations we expect to direct to those gaps.

We have developed a list of ideas for how to significantly increase the money GiveWell directs to our recommended charities, and are planning to work on the most promising ideas over the next few months. For example, we think there may be relatively low-intensity steps to take in areas like podcast advertising; in February, we advertised on a small number of podcasts and plan to do so again based on the cost of running advertisements and the additional donations to top charities we tracked as a direct result of those ads.

We are also trying to hire another Research Analyst, Outreach Focus to expand our capacity to communicate with donors and other individuals who rely on GiveWell’s research.

The post Mid-year update on GiveWell’s progress appeared first on The GiveWell Blog.

This is the third of four posts that form our annual review and plan for the following year. This post reviews and evaluates GiveWell’s progress last year as an organization and sketches out some high level goals for the current year. The first two posts covered GiveWell’s progress and plans on research. The last post in the series will look at metrics on our influence on donations in 2016.

First, a point of clarification. GiveWell as a legal entity currently employs both (a) staff whose work is described on givewell.org (finding outstanding evidence-backed, cost-effective programs) and (b) staff who work on the Open Philanthropy Project. We expect Open Philanthropy to become a separate organization this year (more below), pending board approval. The scope of this post is limited to (a) – the parts of the organization that will not become part of Open Philanthropy. Open Philanthropy has written about its progress and plans in this post.

Below, we first note three high-level points about where GiveWell is as an organization today. We then reflect on four questions that are important for thinking about our performance as an organization:

• Do we have sufficient staff capacity?
• Does our impact justify our operating expenses?
• Does GiveWell have a positive and accurate public image?
• Are we in a stable financial position?

Major organizational developments

Separation of the Open Philanthropy Project

We had aimed to complete the transition of Open Philanthropy staff to a new entity by the end of 2016 and did not accomplish this goal, though we are now effectively operating as two separate teams. We now expect, pending board approval, to complete the legal split by mid-2017. After the split, there will continue to be some shared staff between the organizations (GiveWell staff will track the time they spend on work for Open Philanthropy and GiveWell will bill Open Philanthropy for the time). We will continue to share office space.

GiveWell as an entity currently employs 35 staff members. After the split, we anticipate that GiveWell will continue to employ 15-20 of the current employees and that Elie Hassenfeld will remain as Executive Director of GiveWell. Holden Karnofsky, Co-Founder of GiveWell, currently spends very little time on GiveWell and will work full time for Open Philanthropy.

Outreach is now more of a limiting factor than research

We’ve gone from feeling that we had more funding available than we had good giving opportunities to a situation where we believe that strong giving opportunities have surpassed available funding. We estimate that we left over $100 million worth of very strong opportunities (top charity execution level 1 or 2 gaps, excluding GiveDirectly) unfilled last year.

This is due to increased research output (we added three new top charities and two new standouts) in 2016, an expectation of increased research output in the future (from our standard process and Incubation Grants), and decreased expectations of funding from Good Ventures. In a change from the previous year, Open Philanthropy’s tentative guess is currently that the “last dollar” it will give (from the pool of currently available capital) has higher expected value than gifts to GiveWell’s top charities today, leading it to recommend that Good Ventures cap its giving to GiveWell’s top charities at $50 million in 2016.

We expect to put more emphasis on expanding our outreach to potential donors interested in following our recommendations in 2017 than we have in past years. We are at early stages of thinking through what that might involve.

Organizational maturity

GiveWell will be 10 years old this year and we feel that we’ve reached a relatively stable place in our development. We are now making a major effort to strengthen our organizational infrastructure through filling specialized roles, particularly in operations (finance, donations management, technology, etc.); formalizing policies and procedures; and creating contingency plans for replacing senior staff.

Four key questions

Below we pose and respond to four questions about how we are doing as an organization.

Do we have sufficient staff capacity?

Operations: To date we have not had sufficient capacity for operations and have been slower to make improvements to our systems than we would have liked. In the last year, we have begun to make major changes to GiveWell’s operations team to try to correct for this. Sarah Ward was named Director of Operations, a new role, and we are pursuing a strategy of (a) hiring specialized firms to handle more of the HR and IT work that generalist staff have done in the past; (b) replacing our external accountants and auditors with firms that specialize in non-profits; and (c) moving current staff into and hiring for specialized roles, such as a donations manager, donor relations assistant, controller, and office manager. Our number of generalist operations staff has decreased; we expect to continue to have a need for a small number of generalist staff to manage relationships with external firms and fill gaps between specialist domains.

Our current operations team includes a Director of Operations, two operations generalists (who work on the website, accounting, recruiting, personnel management, donation processing, and IT), an Office Manager, an Administrative Assistant, a Donations Manager, a Donations Assistant, and a Donor Relations Assistant. We are hiring for an Operations and Legal Program Manager and expect to hire for additional roles in the coming months. After the expected spinoff of Open Philanthropy into a separate organization, the office manager, administrative assistant and one of the operations generalists will divide their time between the two organizations and Sarah will manage operations for both organizations temporarily; Open Philanthropy will begin building a separate operations team this year.

Research: Seven staff work on GiveWell’s research full time or close to full time. Elie Hassenfeld, GiveWell’s Executive Director, spends about half his time on GiveWell research. Elie spends the other half of his time on a combination of the Open Philanthropy project (about 20% of his time currently) and overseeing outreach, recruiting, and operations for GiveWell.

Josh Rosenberg and I have taken over much of the research work that Elie and Holden, co-founders of GiveWell, used to do, including all updates on current top charities, reviewing top charity contenders, managing research staff, and some intervention assessments. Holden now spends almost no time on GiveWell research.

We feel that we have sufficient capacity to follow up with our current top charities, consider promising contenders for top charity recommendations, and make decisions about Incubation Grants. We do not yet have sufficient capacity for reviewing the evidence for and modeling cost-effectiveness of interventions. We aim to make at least one hire for this work in the next few months. More on this in our post about our research plans for the year.

Outreach: As noted above, we feel we’ve reached the point where we are identifying outstanding giving opportunities more quickly than we can expand our reach to donors to fill the opportunities. Throughout most of our history, we felt that the opposite was true, that the amount of funding we could influence surpassed the opportunities we had identified, so this represents a significant shift for us. We don’t yet have concrete plans for future outreach work, but expect to give outreach significantly more attention than we have in the past.

We currently have one staff member, Catherine Hollander, who works on outreach full-time. Our outreach priorities in 2016 were to speak or meet with all major donors who were interested in talking to us, take any opportunities that came up to discuss our work with the media, and continue posting regularly to our blog. We feel that we accomplished our goals for connecting with major donors and keeping up with media requests, and fell short on blogging.

Catherine is leading the search for a Research Analyst, Outreach Focus to do more of the types of outreach we’ve focused on in the past, namely connecting with more media and major donors, and increasing the frequency of blog posts.

Does our impact justify our operating expenses?

GiveWell’s impact on donations (or “money moved”) to our recommended charities likely decreased somewhat in 2016. We are in the process of gathering and analyzing data on our influence on donations, but expect it to be in the range of $80-90 million to recommended charities and $9.2 million for Incubation Grants. Money moved to top charities in 2015 was $110 million.

Good Ventures’ giving to top charities fell from about $70 million to $50 million, due to changes in the way it is allocating funding across priorities and to a large one-off grant to GiveDirectly in 2015. Based on GiveWell’s recommendations, Good Ventures also funded $9.2 million in Incubation Grants, up from about $400,000 to $500,000 in each of 2014 and 2015.

Over the same period, we spent approximately $2 million on our operations. In total, GiveWell as an entity spent about $5.5 million on operational expenses, of which $3.5 million was spent on the Open Philanthropy Project.

We previously wrote that we believe that expenses that are 15% of money moved are well within the range of normal, so we feel comfortable with the relative size of our operating expenses at this point.

Does GiveWell have a positive and accurate public image?

We believe that GiveWell’s public image is largely positive and reasonably accurate. This is true for all or nearly all of the major media coverage we have received. See, for example, coverage on NPR and in The Atlantic, Esquire and Vox.

There are two aspects of our public image that we would like to change. First, media has sometimes portrayed our top charities as having guaranteed impact and as being the “best” charities—for example, a 2015 article in The Atlantic said, “If what you want is to save lives with certainty, several people said, you have to go to GiveWell.” We believe that our top charities offer the highest expected value among evidence-backed opportunities that we have found to date, but are not risk-free and may not be the best giving opportunities for donors with different values or unique expertise, connections, or resources. Second, charities may have an inaccurate view of the costs and benefits of engaging with us—more in this post.

Our biggest public image project in the last year was launching a redesigned website. This project took much longer than expected. The original launch date was April 2015, but due to unexpected problems and lack of staff capacity, it didn’t go live until September 2016. Our previous website had an outdated look and confusing architecture. We think the new one is a large improvement, though we aim to make some further improvements in the future.

Are we in a stable financial position?

The short answer is yes.

In 2016, we raised about $3 million in revenue available for funding our operations that was not specifically for funding Open Philanthropy Project expenses (Open Philanthropy has, recently, been fully funded by Good Ventures). We have roughly projected GiveWell’s expenses (excluding pre-split Open Philanthropy expenses) at $2.7 million in 2017 and $3.2 million in 2018. Given our money moved to top charities and our experiences with fundraising in the past, it seems reasonable to expect that we will be able to raise this funding, though we expect to do a more detailed analysis of our financial situation once the details of the split with Open Philanthropy have been fully worked out.

We do not expect revenue available for operations to decrease as a result of splitting with Open Philanthropy because most major donors have told us that they support GiveWell due to our work identifying top charities. We think it is likely that Good Ventures will continue to support 20% of GiveWell’s operational budget, as it has for the last several years.

The post GiveWell as an organization: progress in 2016 and plans for 2017 appeared first on The GiveWell Blog.

This is the second of four posts that form our annual review and plan for the following year. The first post reviewed our progress in 2016. The following two posts will cover GiveWell’s progress and plans as an organization and metrics on our influence on donations in 2016.

Our primary research goals for 2017 are to:

• Speed up our output of new intervention assessments, by hiring a Senior Fellow and by improving our process for reviewing interventions at a shallow level.
• Increase the number of promising charities that apply for our recommendation. Alternatively, we may learn why we have relatively few strong applicants and decide whether to change our process as a result. Research Analyst Chelsea Tabart will spend most of her time on this project.
• Through GiveWell Incubation Grants, fund projects that may lead to more top charity contenders in the future and consider grantees No Lean Season and Zusha! as potential 2017 top charities.
• Further improve the robustness and usability of our cost-effectiveness model.
• Improve our process for following the progress of current top charities to reduce staff time, while maintaining quality. We also have some specific goals (discussed below) with respect to answering open questions about current top charities.

We discuss each of these goals in more depth below.

Intervention assessments

Intervention assessments are key to our research process. We generally only consider recommending funding for programs that are implementing one of our priority programs (an exception is if an organization has done rigorous evaluation of its own program, though in practice we have found this to be very rare). In recent years, we have completed few intervention reports, which has limited our ability to consider new potential top charities. We plan to increase the rate at which we form views on interventions this year by:

• Hiring a Senior Fellow (or possibly more than one). We expect a Senior Fellow to have a Ph.D. in economics, public health, or statistics or equivalent experience and to focus on in-depth evidence reviews and cost-effectiveness assessments of interventions that appear promising after a shallower investigation. In addition, Open Philanthropy Project Senior Advisor David Roodman may spend some more time on intervention related work.
• Doing low-intensity research on a large number of promising interventions. We generally start with a two to four hour “quick intervention assessment,” and then prioritize interventions for a 20-30 hour “interim intervention report” (example). We don’t yet have a good sense of how many of these of these we will complete this year, because we’re unsure both about how much capacity we will have for this work and about how many promising interventions there will be at each step in the process.
• Continuing to improve our systems for ensuring that we become aware of promising interventions and new relevant research as it becomes available. We expect to learn about additional interventions by tracking new research, particularly randomized controlled trials, in global health and development and by talking to select organizations about programs they run that they think we should look into.

Charity applications

In the past few years, we have been surprised by how little interest there has been from charities in applying for a GiveWell recommendation. Our impression is that for global health and development charities there are relatively few funders of our size: in 2015, we tracked $110 million given due to our research; we are in the process of compiling the data for 2016, but expect it to be in the range of $80-90 million to recommended charities. We would like to better understand whether we have failed to get the word out about the potential value we offer or communicate well about our process and charities’ likelihood of success, or, alternatively, whether charities are making well-informed decisions about their fit with our criteria. (More on why we think more charities should consider applying for a GiveWell recommendation in this post.)

This year, we have designated GiveWell Research Analyst Chelsea Tabart as charity liaison. Her role is to increase and improve our pipeline of top charity contenders by answering charities’ questions about our process and which program(s) they should apply with, encouraging promising organizations to apply, and, through these conversations, understanding what the barriers are to more charities applying.

We aim by the end of the year to have a stronger pipeline of charities applying, have confidence that we are not missing strong contenders, or understand how we should adjust our process in the future.

Incubation Grants

We made significant progress on Incubation Grants in 2016 and plan in 2017 to largely continue with ongoing engagements, while being open to new grantmaking opportunities that are brought to our attention.

Among early-to-mid stage grants, we plan to spend the most time on working with IDinsight and New Incentives (where our feedback is needed to move the projects forward), and a smaller amount of time on Results for Development and Charity Science: Health (where we are only following along with ongoing projects).

Another major priority will be following up on two later-stage grantees, No Lean Season and Zusha!, groups that are contenders for a top charity recommendation in 2017. For No Lean Season, a program run by Evidence Action, our main outstanding questions are whether the program will have room for more funding in 2018 and whether monitoring will be high quality as the program scales. We have similar questions about Zusha! and in addition are awaiting randomized controlled trial results that are expected later this year.

Cost-effectiveness model

We plan to continue making improvements to our cost-effectiveness model and the data it draws on (separate from adding new interventions to the model, which is part of the intervention report work discussed above). Projects we are currently prioritizing include:

• Making it more straightforward to see how personal values are incorporated into the model and what the implications of those values are.
• Revisiting the prevalence and intensity adjustment that we use to compare the average per-person impact of deworming in places that our top charities work to the locations where the studies that found long-term impact of deworming were conducted. More in this post.
• Improving the insecticide-treated nets model by revisiting how it incorporates effects on adult mortality and adjustments for regions with different malaria burdens and changes in malaria burden over time.

Current top charities

Our goal this year is to maintain the quality of top charity updates while decreasing the amount of staff time we spend and we ask top charities to spend on this work. Below, we detail our plans for following up with each charity.

Deworm the World Initiative, GiveDirectly, and Schistosomiasis Control Initiative (SCI)

We have now followed these groups for several years and do not have major outstanding questions about them. We plan to ask for updates on financial information, monitoring results, and room for more funding and have regular phone calls with them to learn about operational changes that might lead us to ask additional questions.

Against Malaria Foundation (AMF)

We have two major outstanding questions about AMF that we hope to make progress on this year:

1. Will AMF’s monitoring processes be high quality? We wrote about our concerns about AMF’s past monitoring last year and expect new information to be available this year.
2. Going forward, AMF aims to fund larger distributions and commit funding further ahead of when a distribution is scheduled to occur than it has, for the most part, done in the past. Will this increase the extent to which AMF funds displace funds from other sources, or will there continue to be evidence that AMF’s funds are largely adding to the total number of nets distributed? More on this question in our review of AMF.

To help us make progress on these questions, we and AMF have agreed to have monthly calls to discuss questions we have about the monitoring AMF is producing and what AMF is learning about distributions it is considering funding. We will likely also seek out calls with AMF’s partner organizations to discuss these questions.

In order to estimate AMF’s room for more funding, we will seek out information on the location and size of funding gaps for mass net distribution campaigns from AMF, the African Leaders Malaria Alliance, and possibly other funders of nets. As we have in the past, we will use this information in conjunction with conversations with AMF about non-funding bottlenecks to its ability to fill various gaps.

The END Fund – deworming program

Compared with the charities we have recommended for several years, we have more open questions about the END Fund. The main questions we plan to seek more information on this year are:

1. We have not yet seen monitoring on par with that from our other top charities from the END Fund. We expect results from coverage surveys from END Fund programs this year. Will these surveys be high quality and demonstrate that the END Fund is funding successful programs?
2. We have not yet tried to compare the cost-effectiveness of the END Fund to our other top charities in our cost-effectiveness model. We will be seeking additional information from the END Fund about cost per treatment and baseline infection rates.
3. Questions around room for more funding: the extent to which funding due to GiveWell’s recommendation increases the amount that the END Fund spends on deworming versus other programs, actual and projected revenue from other sources, and what deworming grantmaking opportunities the END Fund expects to have.

We visited the END Fund’s programs in Rwanda and Idjwi island, DRC in January 2017 and will publish notes and photos from our visit shortly.

Malaria Consortium – seasonal malaria chemoprevention program

As with the END Fund, we have more open questions about Malaria Consortium than we do for the charities we have recommended for several years. Our main priorities are:

1. Further research on the evidence of effectiveness, cost-effectiveness, and potential downsides of seasonal malaria chemoprevention (SMC) (due to time constraints we have not yet completed a full intervention report, though we felt sufficiently confident in the intervention to recommend Malaria Consortium).
2. Getting a better understanding of the methodology Malaria Consortium uses for estimating coverage rates.
3. Completing a more in-depth room for more funding analysis for the program for 2018 than we did for 2017.

Malaria Consortium expects to have several new studies of its SMC programs to share in April 2017 (details).

We may visit a Malaria Consortium seasonal malaria chemoprevention program in summer 2017.

Sightsavers – deworming program

As with the END Fund and Malaria Consortium, we have more open questions about Sightsavers than we do for the charities we have recommended for several years. We expect to make limited progress this year because the first deworming mass drug administration funded with GiveWell-influenced funds is not expected to take place until September at the earliest and monitoring results aren’t expected until early 2018. Because Sightsavers has done fairly little deworming in the past year, we don’t expect to be able to learn much from its ongoing programs. Our main priorities for the year are:

1. Getting more information from Sightsavers about baseline prevalence and intensity of worm infections in the areas it is working, to inform our cost-effectiveness analysis.
2. Using Sightsavers’ budget for the projects and planned treatment numbers to improve our estimate of the cost per treatment – another input into our cost-effectiveness analysis. Our current cost per treatment estimate is very rough.
3. Completing a room for more funding analysis for 2018.

Standout charities

Standout charities are groups that we have a large amount of information about and that meet some but not all of our criteria. Because we have not followed them closely over time, it is possible that they may now be a stronger fit (or that they no longer focus on the program we reviewed). We plan to have at least one phone call with each of these groups to discuss whether anything has changed that might lead us to reopen consideration of the organization as a potential top charity. Due to our focus on organizations that are most likely to become top charities, we don’t expect to make this work a priority beyond that.

The post GiveWell’s research plans for 2017 appeared first on The GiveWell Blog.

This is the first of four posts that form our annual review and plan for the following year. This post reviews and evaluates last year’s progress on our work of finding and recommending evidence-based, thoroughly vetted charities that serve the global poor. The following three posts will cover our plans for GiveWell’s research in 2017, GiveWell’s progress and plans as an organization and metrics on our influence on donations in 2016.

Summary

We feel that 2016 was a highly successful year for GiveWell’s research. We accomplished or made significant progress on all of our top priorities and accomplished some of the goals we didn’t know if we would have time for. Our research output was greater than in any past year. We added three new top charities and two new standout charities. We made important progress on building the pipeline of future GiveWell top charities through our work on GiveWell Incubation Grants. The research team’s staff capacity has continued to increase and we expect output to continue to grow.

More subjectively, we feel that the quality of our research has continued to improve. Of particular note are our improved understanding of room for more funding for insecticide-treated nets and of the evidence for deworming.

We compare our annual output for 2012-2016 in this spreadsheet.

Our progress in 2016 relative to our plans

In early 2016, we laid out our goals for research in 2016. Below we discuss each goal in two of the categories we set out last year, “top priorities” and “other research we will undertake if we have the time to do so,” and what progress we made on each.

“Top priorities”

Supporting the development of potential future GiveWell top charities: making grants to organizations that could become top charity contenders in the future or supporting research that could lead to more organizations that are a strong fit with our criteria. […]

We investigated and recommended that Good Ventures make grants to five early stage projects: No Lean Season (migration assistance), New Incentives (conditional cash transfers), Zusha! (road safety), Charity Science: Health (immunizations), and Results for Development (childhood pneumonia treatment). We have published grant descriptions for most of these grants; three are forthcoming. We have also begun working closely with IDinsight to partner with charities that work on priority programs to strengthen their monitoring and evaluation to increase the chances that the charities meet our criteria in the future.

Considering additional funding for insecticide-treated nets [beyond the Against Malaria Foundation] […]

We spent relatively little time on this priority because (a) the Against Malaria Foundation (AMF) succeeded in signing agreements for several major distributions early in the year, increasing our estimate of its ability to absorb additional funds; and (b) our initial conversations with large funders of nets seemed unlikely to result in a top charity recommendation. At the end of the year, we found that the Against Malaria Foundation was able to absorb considerably more funding than we directed to it. This was in part due to Good Ventures’s shift away from continued growth in funding for GiveWell top charities. Separately, we also gained a stronger understanding of the size and nature of funding gaps for nets globally through country case studies and conversations to understand the global funding landscape.

Intervention prioritization: quick investigations on a large number of interventions with the goal of finding more priority programs. […]

This was a major priority for us and we made some progress, but not as much as we wanted to. The main things we did were (a) quick reviews (3-10 hours spent reviewing the evidence base) for ~30 programs, which ultimately led us to prioritize seasonal malaria chemoprevention (SMC) and recommend Malaria Consortium’s work on SMC; (b) publishing three “interim intervention reports” (on SMC, integrated community case management, and severe acute malnutrition) which have provided a template for mid-level assessments and enabled more staff to produce such reports in 2017 (one of which, on Sayana Press, has been published); and (c) writing a full intervention assessment of voluntary male medical circumcision, which is now one of our priority programs.

Current top charities: continuing to follow our current top charities and trying to answer our highest priority unanswered questions about these groups. […]

We answered our most important questions about the top charities we recommended in 2015. In particular, we had stronger answers at the end of the year on AMF’s progress at signing agreements, the quality of AMF’s monitoring, Schistosomiasis Control Initiative’s past spending and financial position, and had an overall much stronger understanding of Deworm the World Initiative, particularly its work in Kenya.

However, we feel that we spent too much time on this work in 2016. Three staff spent the majority of their time on top charity updates and, while following the progress and plans of our top charities is a crucial piece of GiveWell’s work, the value we got from this work felt out of proportion with the time spent. In 2017, we plan to have a single staff member do most of this work and expect it to take a half to two-thirds of a full-time job. Three other staff will spend a small portion of their time, totaling approximately the equivalent of one full-time job, on this work.

New evidence on deworming and bednets. The next round of follow up on a key deworming study is expected to be available later this year and could make a big difference to our view of deworming. We’re also looking more into the degree to which insecticide resistance may be reducing the impact of bednets. […]

We completed an evidence review on the impact of insecticide resistance on malaria control. David Roodman, Senior Advisor at the Open Philanthropy Project spent several months revisiting the evidence for deworming and summarizing his findings in two blog posts.

We wrote last year that we expected to see a new round of follow-up data on a key deworming study that could significantly affect our view of deworming. We have seen preliminary results and hope to get more complete results later this year and write about them at that time. We also decided to recommend a ~$1 million grant to support a more intensive 20-year follow-up to the Worms at Work study (writeup forthcoming).

“Other research we will undertake if we have the time to do so”

Micronutrient fortification charities. […]

We completed interim reviews of Food Fortification Initiative and Project Healthy Children and added them to our list of standout charities.

Neglected tropical disease (NTD) charities [and…] other organizations – if organizations apply for a recommendation and seem sufficiently promising, we will aim to review them.

Perhaps the most important development in 2016 came out of two secondary goals for 2016: continuing investigations of deworming programs (which led to recommending Sightsavers and the END Fund for their work on deworming) and continuing to be open to applications from charities (which led to recommending Malaria Consortium for its work on seasonal malaria chemoprevention).

At the same time, in the past few years, we have been surprised by how little interest there has been from charities in applying for a GiveWell recommendation. We have come to believe that charities may have misconceptions about our process or lack the information about whether they would be a fit for our criteria, and that this could be improved by us reaching out to more promising organizations and taking the time to understand the reasons why they have not applied in the past. We discuss what we are doing this year on charity outreach in the next post in this series.

Surgery charities. We have had several conversations with organizations that work on cataract surgery and we may reach out to organizations that work on obstetric fistula surgery. […]

In addition to the conversations with organizations that work on on cataract, we also spoke to several groups that work on obstetric fistula. We have not identified a group we would like to invite to apply. This work is now moving ahead primarily through our engagement with IDinsight to work with charities to strengthen their monitoring and evaluation.

Publishing research we largely completed in 2015: updates on standout charities (GAIN, IGN, and Living Goods), interim reviews of charities we began investigating in 2015 (Sightsavers, END Fund, and Project Healthy Children), and intervention reports (folic acid fortification, surgery for cataracts, trachoma and fistula, measles immunization campaigns, mass drug administration for lymphatic filariasis, and “Targeting the Ultra Poor”).

We made some progress on this goal. We published updates on standout charities (GAIN, Iodine Global Network, and Living Goods), reviews of charities we began investigating in 2015 (Sightsavers, the END Fund, and Project Healthy Children), and one of the intervention reports we hoped to publish in 2016 (cataract surgery). We did not publish the other intervention reports we hoped to (folic acid fortification, surgery for trachoma and fistula, measles immunization campaigns, mass drug administration for lymphatic filariasis, and “Targeting the Ultra Poor”).

Other work

Compared to previous years, we made a lot of progress on improving the usability of and getting staff engagement with our cost-effectiveness model. We designated a staff member, currently Chris Smith, to work on this close to full-time. Chris re-formatted the file to make it easier for staff and others to input uncertain and subjective values, integrated the seasonal malaria chemoprevention analysis into the model, and restructured the bed nets model to take into account country-level variation in malaria rates.

The post GiveWell’s progress on research in 2016 appeared first on The GiveWell Blog.

In addition to evaluations of other charities, GiveWell publishes substantial evaluation of ourselves, from progress against our goals to our impact on donations. We generally publish quarterly updates regarding two key metrics: (a) donations to top charities and (b) web traffic (though going forward, we may provide less frequent updates).

The tables and chart below present basic information about our growth in money moved and web traffic in the first quarter of 2016 compared to the previous two years (note 1).

Money moved and donors: first quarter

Money moved by donors who have never given more than $5,000 in a year increased about 50% to $1.1 million. The total number of donors in the first quarter increased about 30% to about 4,500 (note 2).

Most of our money moved is donated near the end of the year (we tracked 70% or more of our total money moved in the fourth quarter each of the last three years) and is driven by a relatively small number of large donors. Because of this, we do not think we can reliably predict our growth and think that our year-to-date total money moved provides relatively limited information about what our year-end money moved is likely to be (note 3). We therefore look at the data above as an indication of growth in our audience.

Web traffic through April 2016

Growth in web traffic excluding Google AdWords increased 10% in the first quarter. GiveWell’s website receives elevated web traffic during “giving season” around December of each year. To adjust for this and emphasize the trend, the chart below shows the rolling sum of unique visitors over the previous twelve months, starting in December 2009 (the first period for which we have 12 months of reliable data due to an issue tracking visits in 2008).

We use web analytics data from two sources: Clicky and Google Analytics (except for those months for which we only have reliable data from one source). The raw data we used to generate the chart and table above (as well as notes on the issues we’ve had and adjustments we’ve made) is in this spreadsheet. (Note on how we count unique visitors.)

Note 1: Since our 2012 annual metrics report we have shifted to a reporting year that starts on February 1, rather than January 1, in order to better capture year-on-year growth in the peak giving months of December and January. Therefore, metrics for the “first quarter” reported here are for February through April.

Note 2: Our measure of the total number of donors may overestimate the true number. We identify individual donors based on the reported name and email. Donors may donate directly to our recommended charities and not opt to share their contact information with us, or donors may use different information for subsequent donations (for example, a different email), in which case, we may mistakenly count a donation from a past donor as if it was made by a new donor. We are unsure but would guess that the impact of this issue is relatively small and that the data shown are generally reflective of our growth from year to year.

Note 3: In total, GiveWell donors directed $2.6 million to our top charities in the first quarter of 2016, compared to $2.0 million that we had tracked in the first quarter of 2015. For the reason described above, we don’t find this number to be particularly meaningful at this time of year.

Note 4: We count unique visitors over a period as the sum of monthly unique visitors. In other words, if the same person visits the site multiple times in a calendar month, they are counted once. If they visit in multiple months, they are counted once per month.

The post Update on GiveWell’s web traffic / money moved: Q1 2016 appeared first on The GiveWell Blog.

GiveWell is dedicated to finding outstanding giving opportunities and publishing the full details of our analysis. In addition to evaluations of other charities, GiveWell publishes substantial evaluation of our own work. This post lays out highlights from our 2015 metrics report, which reviews what we know about how our research impacted donors. Please note:

• We report on “metrics years” that run from February through January; for example, our 2015 data cover February 1, 2015 through January 31, 2016.
• We differentiate between our traditional charity recommendations, our work on the Open Philanthropy Project, and other charitable giving.
• More context on the relationship between Good Ventures and GiveWell can be found here.

Summary of influence: In 2015, GiveWell influenced charitable giving in several ways. The following table summarizes our understanding of this influence.

Total money moved: In 2015, GiveWell tracked $110.1 million in money moved to our recommended charities. Our money moved only includes donations that we are confident were influenced by our recommendations.

Open Philanthropy Project: As part of our work on the Open Philanthropy Project, we advised Good Ventures to make grants totaling $6.4 million. This was in addition to Good Ventures’ support for our recommended charities.

Money moved by charity: Our four top charities received the majority of our money moved. Our four standout charities received a total of $2.2 million.

Money moved by size of donor: In 2015, the number of donors and amount donated increased across each donor size category. In 2015, 95% of our money moved (excluding Good Ventures) came from about 15% of our donors, each of whom gave $1,000 or more.

Donor retention: The total number of donors who gave to our recommended charities or to GiveWell unrestricted increased about 60% year-over-year to 15,274 in 2015. This included 10,669 donors who gave to our recommended charities for the first time. Among all donors who gave in the previous year, about 40% gave again in 2015, up from about 33% who gave again in 2014.

Our retention was stronger among donors who gave larger amounts or who first gave to our recommendations prior to 2013. Of larger donors (those who gave $10,000 or more in either of the last two years), about 80% who gave in 2014 gave again in 2015.

GiveWell’s expenses: GiveWell’s total operating expenses in 2015 were $3.4 million. Our expenses increased from about $1.8 million in 2014 as the size of our staff grew, largely to support the Open Philanthropy Project. We estimate that about one-third of our total expenses ($1.1 million) supported our traditional top charity work and about two-thirds supported the Open Philanthropy Project. In 2014, we estimated that expenses for our traditional charity work were about $900,000.

Donations supporting GiveWell’s operations: Prior to 2013, GiveWell relied on a small number of donors to provide unrestricted support for our operations. Since 2013, we have asked more donors to support our operational costs and asked donors to support us at a higher level than we had in previous years. In 2015, we raised $4.9 million, up from $3.0 million in 2014. Several institutions and the six largest individual donors contributed about two-thirds of GiveWell’s funding in 2015.

Donor demographics: We continued to collect information on our donors. We found the picture of our 2015 donors to be broadly consistent with previous information. Based on reports from donors who gave $2,000 or more, we found:

• The most common ways that donors found us were via Peter Singer and personal referrals.
• About 70% of our donors are under 40, and about 60% work in technology or finance.

Web traffic: Unique visitors to our website increased by 12% in 2015 compared to 2014 (when excluding visitors driven by AdWords, Google’s online advertising product).

For more detail, see our full metrics report (PDF).

The post GiveWell’s money moved and web traffic in 2015 appeared first on The GiveWell Blog.

Over the past couple of years, we’ve put a lot of effort into hiring and training staff and we now have significantly more capacity to do research than we have in past years. Some of our increased capacity will support the Open Philanthropy Project, which we hope will be a separate organization by the end of 2016; its plans for the year will be discussed on the new Open Philanthropy Project blog. We also expect to have more capacity for GiveWell’s work of finding outstanding evidence-based charities.

At the same time, we have come to believe that the kind of work we’ve recently been doing to find top charities – deeply investigating the most promising-seeming charities we know of, based largely on which interventions they carry out – has limited promise. In past years – and at the beginning of this year – we hoped that these investigations would lead relatively quickly to new top charities. Now, we believe that we’ve already (previously) identified most of the strongest charities by our criteria, and there aren’t many strong candidates left (though there are a few that we continue to investigate, and we remain willing and eager to investigate further promising groups if we come across them). With that in mind, we have begun seeing more potential in other research priorities, such as supporting the development of new organizations and new evidence bases.

A future post will elaborate on why we’ve formed this view. This post focuses on laying out our plans for GiveWell’s research work in 2016, taking this view into account.

In brief, in 2016:

• We plan to focus much of our capacity on a small number of initiatives that are unlikely to result in new top charities in 2016, but which we hope will lead to new top charities that are competitive with our current top charities in 2017 or 2018.
• We plan to intensify our work following our current top charities and are tentatively planning to make site visits to distributions funded by the Against Malaria Foundation and work supported by Evidence Action’s Deworm the World Initiative.
• We are also planning a substantial project focused on the question of whether or not we should recommend that Good Ventures give significantly more than it has in the past to support insecticide-treated nets, arguably the most promising area we know of for substantial additional funding.
• We also hope to take on additional work (described in detail below) but plan to prioritize this work below the items listed above.
• We plan to put more staff time into donor outreach than we have in the past and discuss our priorities for that work below.

This plan represents a significant shift from previous years, when our primary goal was improving the list of top charities we published at the end of each year. We plan to write more about the reasoning behind this shift in a future post.

What we’ve done so far this year

In January and February 2016 we:

• Put significantly more effort into getting input on our plans from non-senior staff than we have in past years. To start, we asked staff and a small number of GiveWell followers to make probabilistic predictions about which charities would become top charities if we investigated them this year. The results of this exercise led to this initial list of possible priorities (listed in no particular order). This list represents the organizations that we would guess are most likely to become top charities at the end of 2016. When we later revised our plan, we held a series of staff meetings to discuss details of the plan and what the new plan might be missing.
• Had exploratory conversations with several charities on that list and others in the field of global health and development. The goal of conversations with charities was to explain our application process and ask them basic questions about their programs, monitoring and evaluation, and need for additional funding. The goal of conversations with others working in global health and development was to generate a list of additional organizations to contact.

Ultimately, this work made us more pessimistic that prioritizing work on all the organizations listed above would lead to new top charities by the end of 2016, as discussed in the introduction of this post, and we refined our plans for the year as a result. A future post will elaborate on this development.

Top priorities for research

Our top priorities are:

• Supporting the development of potential future GiveWell top charities: making grants to organizations that could become top charity contenders in the future or supporting research that could lead to more organizations that are a strong fit with our criteria. This work is unlikely to result in new top charities in 2016, but we hope it will lead to new top charities that are competitive with our current top charities in 2017 or 2018. This work might include:
  • Providing early stage funding to organizations that aim to scale up programs with strong evidence of effectiveness and cost-effectiveness. (For example, New Incentives or Evidence Action’s No Lean Season program.)
  • Funding research on programs that are candidates to become priority programs. (For example, this grant to support research on an incentives for immunization program.)
  • Funding organizations that run priority programs to increase or improve their monitoring, or funding a third party to do this monitoring.
• Considering additional funding for insecticide-treated nets: A significant funding gap exists for insecticide-treated nets, and this gap appears to be as cost-effective an opportunity as any other we have found. This project involves determining whether there are high quality opportunities to provide significantly more funding for insecticide-treated nets than we have in the past. It will involve conversations with the major bednet funders (e.g., Global Fund to fight AIDS, Tuberculosis, and Malaria and the President’s Malaria Initiative) and others familiar with how to identify funding gaps for bednets and what the options are for monitoring distributions. We have also been discussing with the Against Malaria Foundation (AMF) what it would take to quickly scale up AMF’s work. The goal of this work is to identify additional funding opportunities for funding insecticide-treated nets in 2016.
• Intervention prioritization: quick investigations on a large number of interventions with the goal of finding more priority programs. We have looked at many interventions historically, but regularly learn of programs that we do not know very much about.
• Current top charities: continuing to follow our current top charities and trying to answer our highest priority unanswered questions about these groups. More on this below.
• New evidence on deworming and bednets. The next round of follow up on a key deworming study is expected to be available later this year and could make a big difference to our view of deworming. We’re also looking more into the degree to which insecticide resistance may be reducing the impact of bednets.

Other research we will undertake if we have the time to do so

• Micronutrient fortification charities. Last year, we tried but were unable to find compelling evidence that the Iodine Global Network (IGN) or the Global Alliance for Improved Nutrition (GAIN) had successfully contributed to the impact of salt iodization programs (write-ups forthcoming). We also began investigating Project Healthy Children. We may continue some of these investigations this year and have also reached out to the Micronutrient Initiative and the Food Fortification Initiative.
• Neglected tropical disease (NTD) charities. We began investigating Sightsavers and END Fund’s work on deworming last year and may continue with those organizations this year and expand the investigations to cover multiple NTDs. We have reached out to the Carter Center and Helen Keller International (HKI) about their NTD programs. HKI declined to participate at this time.
• Surgery charities. We have had several conversations with organizations that work on cataract surgery and we may reach out to organizations that work on obstetric fistula surgery. Our initial impression from these conversations is that it will be very challenging to understand the impact that these charities’ programs have. We may also consider other surgical interventions (such as trachoma).
• Other organizations. If organizations apply for a recommendation and seem sufficiently promising, we will aim to review them.
• Publishing research we largely completed in 2015: updates on standout charities (GAIN, IGN, and Living Goods), interim reviews of charities we began investigating in 2015 (Sightsavers, END Fund, and Project Healthy Children), and intervention reports (folic acid fortification, surgery for cataracts, trachoma and fistula, measles immunization campaigns, mass drug administration for lymphatic filariasis, and “Targeting the Ultra Poor”).

Research we considered but do not expect to undertake

The following investigations are ones that we considered doing this year but don’t currently expect to get to. This could change if some of the higher priority work turns out to be less promising than expected.

• Mega-charities. We could try to work with one or more large organizations with very diverse programs to figure out how to scale-up work on one of our priority programs.
• Charities that work on programs that are probably more cost-effective than cash transfers but not by a large enough margin that it seems worth highly prioritizing work on them.
  • Voluntary medical male circumcision. We are interested in talking to PSI, the only major organization we know of working on this program, but do not plan to prioritize this program beyond that.
  • “Targeting the ultra poor” or “graduation” programs.
  • Lymphatic filariasis.
  • Incentives for immunization. We previously funded research on this program and have been working with IDinsight on a cost-effectiveness analysis.
• Immunization programs. We have put in a fair amount of work into looking for room for more funding for scaling up immunization programs and have largely failed to find opportunities (2012 write-up; recent example).

More detail on potential further research on current recommended charities

One of our top priorities for 2016 is continuing to follow our current top charities and trying to answer our highest priority unanswered questions about these groups. We moved over $100 million to these groups in 2015 and whether we recommend a similar (or greater) level of support in 2016, and how we recommend allocating funds among them, depends on answering: (1) what is our best estimate of the organization’s impact and cost-effectiveness? and (2) how much room for more funding do they have?

Top charities

In past years, we’ve updated our top charity reviews once a year, in November. This year, we plan to refresh these reviews twice, in June and November. As we have at the end of the year, we expect to reconsider what recommendation we make to donors about how to allocate donations amongst our top charities in June.

Summary of our research plans for each of our top charities (note that the strategy documents were written in February):

• Against Malaria Foundation (AMF). We plan to follow AMF’s progress closely in 2016. Key questions include (a) how quickly is AMF committing funding to new distributions, and (b) can we get a more detailed understanding of how data is collected in pre- and post-distribution surveys. More details here.
• Schistosomiasis Control Initiative (SCI). The amount of time we spend on SCI this year depends on whether we see a significant improvement in the quality of SCI’s financial information (how it has spent funds, how much funding it holds, and projected expenses). If it does not improve, we will likely deprioritize much additional work on SCI. If it does, we would be interested in exploring the research questions detailed here.
• Deworm the World Initiative. Of our top charities, we feel that there is the largest gap between what we could know and what we do know for Deworm the World. In particular, we’ve focused on Deworm the World’s work in India, because in the past most unrestricted funds were used in India. Going forward, unrestricted funds will largely be used in new programs. We aim to follow Deworm the World’s progress in new countries closely and to better understand its past work by learning more about its program in Kenya. Details here.
• GiveDirectly. Our main goals from following GiveDirectly are to see if the quality of monitoring remains high, it is able to enroll new recipients quickly, and we can learn more about the impact of its work with partners to make cash a baseline against which other development programs are judged. Details here.

Standout charities

• Development Media International. We’re not planning to consider DMI as a possible top charity in 2016. The results from a randomized controlled trial (RCT) of its program that DMI shared last year were not in line with what we would have wanted to see for DMI to become a top charity. More recently, DMI shared some additional results from the RCT (which are not yet public). We believe that taken together these results provide conflicting evidence for DMI’s impact. DMI stands out for its commitment to transparency and rigorous evaluation and we will consider working with DMI to continue to build the evidence base around behavior change through mass media. We see this as a long-term project that is unlikely to result in DMI’s being a top charity in 2016.
• Iodine Global Network. We are planning to follow up with IGN about a few case studies that IGN thought might provide additional evidence of its impact.
• The Global Alliance for Improved Nutrition (GAIN) – Universal Salt Iodization (USI) program. We’re not planning to consider GAIN’s USI program as a possible top charity in 2016. We have not been able to establish clear evidence of GAIN successfully contributing to the impact of iodization programs, and think it is unlikely that more work on this will be useful.
• Living Goods. It’s fairly unlikely that we will consider Living Goods as a possible top charity in 2016. We would revisit this if we were to see significant improvements in the rigor of Living Goods’ monitoring or if we significantly changed our cost-effectiveness estimate for its work.

Plans for donor outreach

We have not historically prioritized outreach at GiveWell, instead choosing to devote staff capacity primarily to our research work. Now, with the addition of new research staff as well as the continued growth of GiveWell’s donor base, we feel it is appropriate to dedicate more capacity to outreach for GiveWell in service of our mission to make our research available to help individuals decide where to give.

In 2016, we plan to have 1.5 staff members devoted to outreach related to GiveWell and the Open Philanthropy Project. Due to this being early on in our outreach work, we’re tentatively planning to reassess our priorities every month for the first half of the year, and then every quarter. As of the publication of this blog post, we expect the following to be top priorities for GiveWell outreach in 2016:

• Donor calls and meetings. We expect that connecting with individuals who have donated to GiveWell will be an important part of our outreach going forward, although as we’re relatively new to prioritizing this, we plan to survey donors about whether this is something that they find useful. We’re hoping to learn more about the donors who use our work and any questions or feedback they have, as well as to offer an opportunity for donors to stay up to date on GiveWell’s work. More here.
• Launching a redesigned website. The redesign will largely improve the look and feel of the site with some minor improvements in navigation and content organization.
• Improving GiveWell’s written communications. This includes:
  • Revisiting and refreshing content on our website (e.g., a recent update to our criteria page) to ensure it’s up-to-date and clearly presented, particularly for individuals who aren’t familiar with our research.
  • Publishing content to our blog, in the hopes of highlighting research and providing additional insight into our values, process, and findings. We will need to put more effort into writing blog posts in order to maintain our previous pace of about one blog post per week, since many types of blog posts that previously appeared here will now be appearing on the Open Philanthropy Blog.

The post GiveWell research plans for 2016 appeared first on The GiveWell Blog.