Note: AMF has reviewed a draft of this post. We believe AMF’s willingness to let individual donors follow along with it – both the successes and the struggles – is highly unusual in the charity world, and an extremely important and positive quality.

Since naming the Against Malaria Foundation (AMF) as our #1 charity in late 2011, we have tracked $10.6 million in donations made to it as a result of our recommendation. In that time, AMF has held the funds while attempting to negotiate a net distribution to spend them on. It has not yet finalized a distribution large enough to spend the bulk of these funds (though it has funded a smaller-scale (~$1 million) distribution in Malawi).

We have been following its negotiations, and this post discusses why AMF has been unable to finalize a sufficiently large distribution. At this time, we plan not to recommend more donations to AMF until and unless it commits the bulk of its current funds to net distributions. Note that this decision does not reflect a negative view of AMF, but rather reflects room for more funding related issues. We note that AMF does not share our view on a lack of room for more funding. AMF’s take is available here. 

In the course of investigating and discussing why AMF was unable to finalize a distribution, we have noted some of what we believe to be mistakes on AMF’s part (more below). We are unsure of how serious these mistakes are, and they are not the driver of the change in recommendation status. We stand by our earlier decision to recommend AMF as our #1 charity, and if and when AMF commits the bulk of its current funds, we will revisit its status and likely recommend it again.

AMF’s response: We disagree with significant parts of GiveWell’s analysis. It is straightforward to find people who want and need nets. We could distribute all the nets we can buy if we were prepared to tolerate high rates of theft. We feel a strong duty to our donors to wait until we can use the funds without significant net losses. This means working carefully with distribution partners who share this goal.

Until recently, it looked as though a finalized distribution might be imminent. Since AMF’s prospects for a distribution in Sierra Leone fell through in September 2013, both we and AMF have been intensifying our efforts to diagnose potential problems, and so much of the content in this post reflects information that is new to both us and AMF.

The rest of this post discusses the following:

• Why has AMF been unable to finalize a sufficiently large distribution? AMF’s past distributions have been relatively small in scale, compared to the types of distributions it is looking at now (the need to increase scale is driven by the increase in funding that GiveWell’s top recommendation has brought about). To find a sufficiently large distribution requires negotiating with the national malaria control programs of countries in sub-Saharan Africa, which we perceive to have some discretion in which funders they work with, and which we perceive to be choosing funders based on a variety of factors including size and reporting requirements. It also requires coordinating with other major funders. AMF has substantial reporting requirements, yet in some cases is able to fund only a relatively small piece of a given country’s distribution. As such, there may be fundamental reason for other parties to the distribution to prefer not to work with AMF if they can obtain nets from another source. We believe that this is the most important and fundamental explanation. We also believe that other factors AMF controls, and which it should have handled differently, may have played a role (see two points down). More
• Does this indicate a lack of global funding gap for insecticide-treated nets? We don’t believe that the struggles to date were a function of the lack of a global funding gap. Over the time period in question, our analysis – based on population-at-risk as compared to number of nets distributed – continued to show a sizable gap; several of AMF’s negotiations proceeded to advanced stages; and our understanding from the conversations we’ve had is that AMF’s negotiations proceeded as far as they did because of an acute desire for more nets. Going forward, we believe that the global funding gap may be smaller, which may steepen AMF’s challenges in terms of negotiating distributions. More
• Has AMF made mistakes? What could it have done better? In 2012, both we and AMF agree that AMF did not investigate enough possibilities at once, and we believe that it improved on this front in 2013. This year, the picture is less clear, as AMF has seemed very close on more than one occasion to having a distribution finalized.
  • AMF states – and we would guess – that the most important factors have been AMF’s relatively small size (compared to other net funders) and its relatively strong reporting requirements (which we broadly support). Broadly speaking, we would not characterize either of these fundamental factors as mistakes, with a caveat (see next bullet point)
  • While some aspects of AMF’s strong reporting requirements are an important part of its value added, other aspects of its requirements may have been excessive. More details below.
  • We believe that AMF’s preference for funding nets (as opposed to other costs such as shipping and evaluation) may have played a role. We see some justification for this preference but also believe that we are – at least partly – in persistent disagreement with AMF on the matter.
  • Finally, we believe that AMF’s communication style has been problematic and that it would benefit from investing more time in building relationships in the malaria control community. The level of reporting sought by AMF makes it likely to meet resistance from organizations with different standards, creating a challenge that we think AMF should have handled better (and is taking steps to handle better in the future).

  More

• How is AMF responding to the distribution challenges? We believe AMF is taking the most appropriate action under the circumstances, which is to solicit a greater degree of involvement from some of the highly experienced people on its Malaria Advisory Group (MAG). We believe that these people will bring an outside perspective, credibility, and experience with the malaria community, and we are optimistic that they will be able to diagnose and respond to AMF’s obstacles. In addition, at AMF’s CEO’s encouragement, some of AMF’s trustees – particularly Peter Sherratt and Richard Lane, who he feels have relevant experience of working with governments and in development – are planning to become more involved in investigating and negotiating potential distributions. This may further help to address problems related to communication style. More
• What will happen to the money donated to AMF if AMF can’t finalize a distribution? AMF remains committed to spending its funds on one or more distributions, and if necessary will distribute nets in multiple smaller distributions over a longer period of time. We asked AMF what its plans would be if this approach too proved infeasible (which we consider to be unlikely but possible). AMF responded that the probability of this is too low to merit planning for this given the continuing prevalence of malaria and a significant global net gap. More
• What is GiveWell’s current view of AMF? Why is GiveWell planning to withdraw its recommendation? We feel AMF has made mistakes and remain in disagreement on some points (more below), but we continue to feel overall that AMF is an outstanding organization in terms of aiming to bring a unique level of accountability to one of the most proven, cost-effective interventions that exists. We are aware that this emphasis on accountability inherently involves challenges when dealing with partners who do not share the same vision of appropriate data collection and reporting. We are still hopeful that AMF will negotiate one or more suitable distributions, and we believe that continuing to hold the funds it has is important to maintaining its negotiating position. More. However, given the significant amount of time that has passed, it now seems to us that there is a substantial risk that AMF will not accomplish this in the near future, and that having more funds to commit would steepen rather than lessen its challenges. (It’s possible that AMF’s challenges would be lessened if it were to receive a very large amount of money – in the range of $30-40 million, which would allow it to fund an entire country’s distribution on its own – but we believe that donations in the range of our projected money moved will increase the number of distributions AMF needs to find while still not making a big enough difference to its size to improve its negotiating position substantially.) Therefore, we believe it is most prudent to donate elsewhere until and unless AMF commits the bulk of the funds it currently holds, at which point we will revisit its ranking.
• Did GiveWell err in its previous recommendation of AMF? What has GiveWell learned from AMF’s struggles? We feel that our prior recommendation of AMF was appropriate, and that the struggles we’ve seen have come mostly from previously recognized (though underestimated) risks. However, we feel that our investigations and communications could have been better, and we have updated our views on a number of fronts. Specifically, (a) we should think harder and communicate more clearly about cases in which our recommendation could bring about a change in the scale of a charity, and the potential difficulties associated with this; (b) we should put more effort into ensuring clear communication between the donors coming to our website and the charities receiving funds from them, in terms of donor preferences about how their funds are to be spent; (c) we should put more effort into interviewing people and organizations who have worked with the organizations we’re investigating, as part of our broader efforts to evaluate people and organizations holistically; (d) we need to be highly cognizant – and to effectively communicate – that as of today, there don’t appear to be any “sure bets” with quantifiable outcomes in charity, especially when it comes to directing relatively large amounts of money. More

This post is partly based on conversations we had over the last few weeks in order to investigate the key factors in AMF’s struggles to negotiate distributions. Some of these conversations remain confidential, but we have posted notes from Don de Savigny of AMF’s Malaria Advisory Group.

Why has AMF been unable to finalize a sufficiently large distribution?Prior to becoming GiveWell’s top-rated charity, AMF funded a number of relatively small distributions carried out by nonprofit organizations. Its largest distribution was the one that was in progress when we first upgraded its ranking to #1: a 270,000-net distribution in Malawi with Concern Universal as a partner, which we wrote about in our March 2012 and September 2012 updates. Since September 2012, AMF has had a total of approximately $9 million in revenues, of which we believe approximately $7 million can be attributed to GiveWell’s recommendation, and the amount it now holds could pay for about 4 million nets – substantially more than it has distributed in the past. Accordingly, AMF has sought to partner directly with a government on a large-scale net distribution, and has not yet finalized such a distribution.

We have discussed the details of negotiations with AMF, but the extent to which this information can be made public is limited. AMF’s future distribution tracker shows the state of possible distributions as of June 2013, and we have discussed some additional detail in previous updates (September 2012, February 2013).

Generally, it seems to us that a given government is, at any given point in time, deciding between multiple options for providing nets for a distribution (including forgoing enough support to do a complete distribution, and instead leaving some areas uncovered). When deciding whether to work with a funder, the following factors all may come into play:

• The size of the funder. President’s Malaria Initiative (PMI) and The Global Fund to Fight AIDS, Tuberculosis and Malaria (GFATM) are particularly large funders that can potentially fund the bulk, or entirety, of a country’s net needs, and so working with them is likely to be particularly worthwhile. While AMF’s level of funding is too large to continue focusing exclusively on distributions of the size it has funded three times in Malawi (several hundred thousand nets per distribution), it is only large enough to fund the bulk of the nets a country needs when dealing with countries with gaps of ~4 million nets or fewer.
• The funder’s requirements in terms of data collection and reporting. A major part of AMF’s appeal, according to our criteria, is its unusual interest in publicly sharing as much data as possible, particularly regarding successful delivery and usage of nets and any changes in malaria case rates pre- and post-distribution. However, AMF’s requests for such data may lessen its appeal for major potential partners for a variety of reasons – in particular, (a) costs and administrative hassles involved with collecting and releasing the data; (b) potentially limited capacity for data collection; (c) potential embarrassment from making the data public (in case the data does not give a favorable picture of how the distribution went).
• The funder’s other requirements and requests about how the distribution is to be executed. In particular, AMF has offered to pay for nets but has generally not offered up front to pay for non-net costs – including shipping, distribution and data collection – which is something we discuss more below.
• Overall degree of comfort and familiarity with the funder. AMF is new to partnerships of this nature, which may reduce the degree of comfort. In addition, AMF may have caused some interpersonal tension, discussed more below.

We aren’t able to disentangle these factors with confidence and say which have been the major ones in AMF’s difficulty finalizing a distribution.

AMF believes that the primary factor is its concern over theft and its reluctance to provide nets without strong measures in place to ensure that they will arrive safely and be distributed to the right people. It believes its insistence on transparency and detailed accountability has caused it to encounter significant resistance from some others in the aid community who have less focus on this area. We are not as confident that we can identify the primary factor, though we do believe that AMF’s emphasis on transparency and accountability (which we broadly support) creates inherent challenges.

Does this indicate a lack of global funding gap for insecticide-treated nets?In the past, we have analyzed population-at-risk data and distribution data to estimate the global net gap, concluding that it is quite large. We are currently in the process of finalizing our refresh of this analysis, and we believe that we will again project a gap, although the fact that nets have gotten cheaper over time – as well as increased funding from other sources – means the estimated gap is likely to be substantially smaller than previous estimates.

We believe that the total number of nets available over the past couple of years has been insufficient to cover the full population at risk. In addition, the partial progress AMF has made in arranging distributions (see notes from Don de Savigny for more) does not, to us, imply a world in which nets are easily available. AMF has encountered multiple countries in which the available funding did not seem sufficient to cover the full planned distribution, and in which AMF was requested to fill the gap.

Going forward, we believe that the global funding gap may be smaller (more), which may steepen AMF’s challenges in terms of negotiating distributions, though it appears that some degree of funding gap will remain.

Has AMF made mistakes? What could it have done better?As stated above, it’s difficult to disentangle the factors that have led to large-scale distributions’ not going forward so far. In some cases we feel that these factors are unavoidable (AMF’s size) and/or reflect AMF’s strengths from our perspective (its interest in collecting and publishing data on distributions). We also wish to note that AMF has completed many smaller-scale distributions (up to several hundred thousand nets) in the past, and we feel that its struggles can fairly be attributed to the increase in size that has come with our recommendation. With that said, there are some cases in which we feel that AMF has made mistakes, and/or in which we continue to disagree with AMF. Much of the thinking here has been driven by recent conversations with people who have worked with AMF; some of these conversations remain off the record, but we have posted notes from Don de Savigny of AMF’s Malaria Advisory Group.

Excessive requirements

Broadly, we support AMF in its attempts to secure strong and public reporting on its distributions, even though doing so may make it more difficult for such distributions to be negotiated.

With that said, it appears to us that AMF’s requirements should have been rethought and adjusted more than they were, at the point when AMF made the transition from partnering with nonprofits to partnering directly with governments. Don de Savigny of AMF’s Malaria Advisory Group was among the people who noted this concern. We believe that AMF now plans to adjust its approach.

A couple of specific notes:

• While we have previously been enthusiastic about AMF’s requests for malaria case rate data, we now believe that the costs of requesting this data may outweigh the benefits. Our view has changed because (a) we have come to believe that this data is highly unreliable (more at our conversation notes from August 15th); (b) we have been told that some countries simply do not have the capacity to collect this data, at least without funding to support their health-system infrastructure more broadly. AMF does believe that if possible an attempt should be made to track the impact of nets in this way, but also recognizes the case for flexibility on this point.
• We still believe that other data AMF seeks (regarding delivery, condition and usage of nets) is valuable and important, and its raising questions around such data may already have had some impact on discussions in the malaria control community (more at our conversation notes from Don de Savigny of AMF’s Malaria Advisory Group).

Preference for funding only nets

AMF prominently states that donations from the public are used exclusively for buying nets. It believes that it must make good on this promise.

Last year, we became concerned that AMF’s preference for funding only net purchases (and not other costs associated with distributions, such as shipping, monitoring and evaluation) might be an obstacle to distributions’ moving forward. We spoke with Rob Mather about the matter, and got the impression in our conversation that (a) this had not been a major obstacle to moving forward; (b) if it became such an obstacle, AMF would raise money specifically for this purpose. Good Ventures’s 2013 donation was specifically marked as available for non-net costs; Rob was reluctant to mark other donations in this way and stated that he did not see a need, and we didn’t perceive it as a major issue, so we didn’t pursue the matter further. We also note that AMF has paid for non-net costs in its Malawi distributions, so it is clear to us that its reluctance to fund such costs is not absolute.

This year we have encountered one case of AMF’s offering to fund additional nets, with another funder’s then arranging to reallocate their funding from nets to non-net costs, specifically net use monitoring costs. (AMF did not feel able to offer funding for non-net costs when it was not funding nets.) This has highlighted to us that we previously underestimated the degree of AMF’s reluctance to fund substantial non-net costs, and with it, the potential consequences.

AMF has stated strongly to us that it does not believe its not offering to fund non-net costs is a major factor in agreements’ not being finalized. We remain unsure of whether AMF’s reluctance to fund non-net costs is a major factor (or any factor) in its struggles to finalize a distribution, but it seems to us that this is a strong possibility.

Having spoken further with AMF, we’ve been given two arguments in favor of avoiding paying for non-net costs:

1. AMF has no in-country presence to effectively monitor non-net costs. It is easier to track and audit the use of funds (in a way that e.g. avoids bribery) in the case of net purchases than in the case of other costs (particularly costs of distribution and evaluation). AMF has the technical capacity for the former, but not for the latter, for which it believes in-country presence or a highly accountable in-country partner is vital. AMF seeks to ensure that major (i.e., government) funders with substantial in-country staff take responsibility for tracking and auditing non-net expenses, and believes that the most practical way to do this is to ensure that major funders allocate their funds to non-net costs. This issue may also have legal implications: AMF has received professional audit guidance that paying for substantial non-net costs could incur legal requirements to audit the funds that would go beyond AMF’s technical capacity. We are unsure of how significant an issue legal requirements represent, and plan to look into the matter further; overall, we believe this could potentially be a strong argument for behaving as AMF has, but remain uncertain. Note that this argument pertains to tracking expenditures and ensuring appropriate use of funds, which is different from evaluating e.g. usage and impact of nets (an area in which we feel that AMF has higher standards than government funders).
2. Many of AMF’s donors prefer that their funds be restricted in this way, and because of the commitments given on AMF’s website, AMF has an obligation to assume by default that this is what its donors prefer. (AMF also notes that a) it believes it has paid for non-net costs when not doing so would hold up a distribution and the other pieces are in place; b) it has approached larger donors, and is prepared to approach other large donors, to ask whether specific donations could be allocated to non-net costs if such a need arose.) In response, we have communicated strongly to AMF that we do not believe the bulk of GiveWell-attributable donations should be restricted in this way, and we have offered to survey donors we have contact information for if it would help resolve the matter.

At this point, we believe there may be good justification for AMF’s reluctance to fund non-net costs, but we also believe that at least part of the reluctance comes from AMF’s perception of donors’ preferences – and thus from a failure on our part to ensure that donors coming from our website communicate their preferences to AMF. We believe that this is at least partially a matter of persistent disagreement: AMF sees value (when communicating to the broader public) in the simplicity and tangibility of “100% buys nets,” while we believe this is not valuable where GiveWell-directed donors are concerned.

Problematic communication style

AMF has pushed for what we see as new levels of information and public accountability, and this creates natural challenges and calls for building relationships in the development community. To this point, we believe that AMF has handled these challenges suboptimally, in the following respects:

• AMF has done relatively little in-person interaction with potential partners, as opposed to communication by email and phone.
• AMF has had a very direct communication style, and may have failed to pick up on (and encourage) communications from people whose style may be less direct. We feel that this dynamic may have led to AMF’s failing to detect potential problems as quickly as it could have – in particular, we were told that AMF should not have been as surprised as it was by the Sierra Leone distribution’s falling through. The sources from which we’ve drawn this view unfortunately remain confidential.

We recognize that the same quality that has made AMF particularly easy for us to communicate with (a highly direct communication style) may be a drawback in its relations with some potential partners.

We believe that AMF is taking appropriate steps (more below) to improve on this front.

The upshot of the above three factors is that AMF may have been perceived as overly demanding by potential partners – coming in as an outsider, asserting substantial requirements without offering to directly reimburse the costs of its requests, and appearing less open to negotiation than it was in reality.

With that said, we doubt that the above factors have been the primary explanation for AMF’s struggles to finalize a distribution. We see its small size and emphasis on strong and transparent data sharing as fundamental obstacles, and we’re cognizant that it’s been willing to negotiate on other factors. We believe that the above issues have largely been a function of transitioning from one sort of partner (nonprofits on smaller-scale distributions) to another (government partners on large-scale distributions), and as discussed below, we believe that AMF broadly is taking appropriate steps to improve on these matters.

How is AMF responding to its difficulties?We believe that Rob Mather, AMF’s Founder and CEO, is at least as eager to see a distribution finalized as we are, and is highly concerned about the delays.

When he was unable to finalize a distribution last year, he committed to building a bigger pipeline of potential partners (details), and we believe he followed through well on that goal, as he explored a significantly larger number of options in 2013.

This year, the main step Rob is taking in reaction to the delays is to solicit greater involvement from his Malaria Advisory Group, specifically Don de Savigny, Bob Snow, and Abdisalan Mohamed Noor. The Malaria Advisory Group normally provides input on AMF’s distributions (examples), but is not employed or paid by AMF, and the individuals in question are well-credentialed malaria scholars (see bios for Don de Savigny, Bob Snow and Abdisalan Mohammed Noor).

We believe this is the appropriate action to take. These individuals are likely to command respect both from Rob and from the malaria community at large, and they will hopefully consider both AMF’s interest in strong reporting and arguments from the malaria community about what level of reporting is excessive, without having strong incentives to put more weight on one or the other. Therefore, their findings are likely to have the best combination of neutrality and respect from both parties that seems feasible in this situation.

Rob has also encouraged several of AMF’s trustees – particularly Peter Sherratt and Richard Lane, who he feels have relevant experience of working with governments and in development – to become more involved in investigating and negotiating potential distributions.

Finally, we have provided written feedback to AMF and its advisors and trustees, and will be sending them this post as well.

Ultimately, we expect AMF to reach reasonable resolutions on its reporting requirements and on the issue of how best to communicate them. We are less sure that we will end up in sync with AMF on the question of non-net costs, but we believe there are legitimate arguments for its perspective on this matter and will respect whatever it ultimately decides on this front, while making clear that AMF should not consider donor preferences to be a major argument for restricting its funds (at least concerning donors going on GiveWell’s recommendation).

What will happen to the money donated to AMF if AMF can’t finalize a distribution?We have encouraged unrestricted donations to AMF, so in most cases we feel it is ultimately up to AMF how to use the funds. (AMF believes it is bound by its public commitments to use the funds to buy nets unless specifically agreed otherwise.) Our impression is that AMF will:

• Commit the funds to a country-wide distribution that meets high standards of data collection and sharing.
• Distribute nets in multiple smaller distributions over a longer period of time.

In either case, we feel that the substantial delay in committing funds represents a substantial negative development, though we also feel that dollars spent well with a delay (as we believe they will be in either case) are better than dollars spent poorly.

We asked AMF what its plans would be if this approach too proved infeasible (which we consider to be unlikely but possible). AMF responded that the probability of this is too low to merit planning for this given the continuing prevalence of malaria and a significant global net gap.

What is GiveWell’s current view of AMF? Why is GiveWell planning to withdraw its recommendation?We continue to feel overall that AMF is an outstanding organization in that it is aiming to bring a unique level of accountability to one of the most proven, cost-effective interventions that exists.

However, a sizable risk has emerged that AMF will not be able to find enough distributions to commit the funds it has, and increasing that amount would steepen this challenge (though perhaps not if AMF raised such significant new funding that it became comparable in size to other major funders in relevant countries). Therefore, we are planning to recommend that people looking to give now donate elsewhere, until and unless AMF demonstrates an ability to put together large enough distributions to commit the funding it has now. We note that AMF does not share our view on a lack of room for more funding. AMF’s take is available here. 

We feel that AMF has made some mistakes, noted above. We are somewhat concerned that AMF is perceived as an “outsider” and – through some combination of communication style and perceived reluctance to fund non-net costs – has been making a poor impression on potential distribution partners. We believe that AMF is taking appropriate steps to diagnose and address these issues, despite some potential disagreements, particularly on the question of non-net costs. We remain unsure of the relative importance of AMF’s mistakes and the fundamental challenge of asking for unusual levels of transparency and accountability (and note that AMF feels the latter is the primary factor).

We wouldn’t guarantee that AMF would immediately become recommended again upon committing the bulk of its funds. The findings that come out of its Malaria Advisory Group’s increased involvement, and the exact circumstances under which a distribution is finalized, would affect our view. However, if and when AMF does commit the bulk of its funds, we will reconsider its ranking and will likely recommend it again.

Did GiveWell err in its previous recommendation of AMF? What has GiveWell learned from AMF’s struggles?We’ve been reflecting on this episode and trying to draw lessons from it. While we still have some reflecting to do, at this point some lessons we believe we can take away are:

• We should think harder and communicate more clearly about cases in which our recommendation could bring about a change in the scale of a charity. We were always aware that our recommendation would substantially increase the funds available to AMF, but initially we didn’t recognize that this would fundamentally change the character of the distributions it was partnering on (see our 2011 review). By our 2012 review, it had become clear that ability to finalize a distribution was a major risk factor, but on balance we decided it was a relatively minor one. We think it’s a good thing to substantially increase the funding of outstanding charities, challenging them to reach a new level of scale. Such an endeavor has potential “upside” of turning a limited organization into one with more absorptive capacity, as long as we don’t multiply the funds available by too high a number in too short a time. In the case of AMF, we feel that the increase in funds was appropriate – enough to bring it to a new level of scale, but not much more than that. However, it needs to be strongly noted, both internally and externally, that such a “step up” involves major risk. We don’t feel that we were clear on the nature of the risk here internally or that we communicated enough about it externally. In the future, when we are recommending that a charity be allowed to substantially increase its scale, we will make more effort to note the major risks that this will entail – and this will apply to this year’s top charities as well.
• We need to ensure clear communication between the donors coming to our website and the charities receiving funds from them, in terms of donor preferences. This is the second time that a recommended charity has expressed to us that it assumes – by default – that its donors wish their funds to be restricted in certain ways. (The other case involves GiveDirectly; in that case we had clear communication up front and ensured that our donate page clearly distinguishes between “flexible funds” and “typical funds,” with the latter being more restricted). We strongly prefer to avoid restrictions on donations, and we don’t want our recommended charities to feel bound by donor preference for anything other than optimal positive impact.
• We should look for ways to deepen and improve our evaluations of people. We’ve previously noted that we place some weight on our evaluations of the people running an organization, and we’ve also expressed reservations about doing so (“we’re wary of putting too much weight on informal, intangible impressions that are likely to end up being ‘tests of how similar charity representatives are to us’ rather than ‘tests of how well-suited charity representatives are to do their work.’ It seems easy for evaluations of people to end up being driven by charisma, by similarity to the evaluator, and/or by characteristics that don’t translate well to nonprofit work.”) We still believe that Rob is an outstanding individual, and we don’t want to place too much weight on complaints about his communication style when these could easily be motivated by frustration with other aspects of AMF, particularly its data disclosure requests. We also believe that the concerns on this front are being addressed appropriately. That said, the complaints have highlighted the point (which we previously noted) that our evaluations of people may overweight their similarity to us, particularly in communication style, and that our evaluations can miss substantial ways in which qualities that make a charity good at communicating with us can be unhelpful (or even drawbacks) for its ability to do its work. We want to find other methods of evaluation that can enrich our understanding of this aspect of an organization, such as the idea in the next point.
• We should put more effort into interviewing other organizations and individuals who have worked with our recommended (or potential recommended) charities. We feel that some important considerations have been raised by our discussions with others in the malaria control community about their interactions with AMF. In general, an organization’s reputation among those it works with is important. This may also be a consideration in favor of highly credentialed organization leaders (who are less likely to come across as outsiders); this is something that we haven’t previously placed much weight on.
• In general, we need to be highly cognizant – and to effectively communicate – that as of today, there don’t appear to be any “sure bets” with quantifiable outcomes in charity, especially when it comes to directing relatively large amounts of money. Many of those who write about GiveWell or use it to donate seem to be under the impression that our top charities represent “sure thing” options allowing a donor to save or improve a known number of lives per dollar given. Yet every year brings new observations about the complexity of giving and about the caveats to seemingly straightforward value propositions. We’ve written before (in 2011) that there is no such thing as a “sure thing” in giving, but we think this point merits further discussion, and we plan to discuss it more in a future post.

With all of that said, we don’t regret our prior recommendation of AMF and believe it was appropriate. AMF had a substantial track record of conducting multiple distributions, including one large distribution, with a level of public disclosure and tracking that no other net distribution charity (and few other charities of any kind) can match. Over 2 years, we have directed enough funding to it to multiply its funds available by several-fold, in the hopes that it would be able to operate at a larger scale; given our other options for recommended charities and the upside if this happened successfully, this was a risk worth taking. It still may be a risk that turns out well, as AMF continues to take steps to improve and to overcome its challenges; it was a risk that has led to substantial learning on our part, which has only been possible because of AMF’s strong transparency and direct communications; and even if AMF is not ultimately able to find appropriate distributions, the result will be delayed (not wasted) donations, as discussed above. We continue to strive to improve, but we stand by our earlier decision to recommend AMF as our #1 charity, and there is a strong possibility that we will do so again in the future.

Finally, we’d like to acknowledge AMF’s willingness to be open and transparent throughout this process. It has reviewed a draft of this post, and we believe its willingness to let individual donors follow along with it – both the successes and the struggles – is highly unusual in the charity world, and will prove enormously beneficial in the long run.