I was pretty interested in where core members of the EA community are donating this year (and read with interest the views of GiveWell's staff), so I asked a few people to write up their giving choices with an explanation of why they chose to give where they did. If you'd like to do the same, e-mail me at the usual address or facebook me. This first post is from Eric Friedman, author of Reinventing Philanthropy and long-time GiveWell supporter. -- William MacAskill
When deciding where to give, my starting point is GiveWell’s recommendations. In past years, a majority of my giving has followed GiveWell’s recommendations, but that will be less so this year because of emerging concerns that I have with GiveWell’s approach. I believe GiveWell’s standard of evidence for impact is too high—so high that it is passing over charities that are likely to be higher impact than its recommended charities. GiveWell’s website describes its process as trying to identify opportunities for which they “could draw a maximally confident, linear, quantified link between donations and outcomes, along the lines of ‘$X per life saved’”. That is, they are heavily concerned about minimizing the risk of failure. My preferred criteria would allow more risk-taking in order to have the potential to do more good. GiveWell has acknowledged that accepting more risk is likely to open the doors to higher-impact giving opportunities; that is a significant motivation for GiveWell Labs, though there are yet to be any recommendations from GiveWell Labs.
The four charities I will give to are the Against Malaria Foundation (AMF), Evidence Action (the parent organization of Deworm the World), UNICEF, and GiveWell. The remainder of this post details why I’m giving to each of the organizations I give to as well as why I am not funding GiveDirectly.
AMF was GiveWell’s favorite charity last year. My understanding is that GiveWell removed AMF from its recommendations largely because of a small, though non-trivial probability of not being able to utilize its cash reserves for a mosquito net distribution. Because of how strongly GiveWell has favored AMF in the past, giving to AMF seems likely to have significantly greater impact than SCI and GiveDirectly, but with a slightly larger probability of failure. I’m willing to accept that (as is Giving What We Can).
Evidence Action is the parent organization for Deworm the World and Dispensers for Safe Water; only the first of those two is recommended by GiveWell. Evidence Action is an off-shoot of Innovations for Poverty Action (IPA), which specializes in doing randomized controlled trials to discover what works to help the world’s poor. IPA does much of the research that GiveWell relies on—even being involved with the research on GiveDirectly. IPA are experts in developmental research and test numerous programs, so I assign a lot of credibility to their putting Dispensers for Safe Water out for expansion.
GiveWell seemed less excited about the evidence for Dispensers for Safe Water, questioning the robustness of the studies evaluating it. That’s a reasonable view, but I’m excited about the possibility of expanding the intervention as a way to get more evidence. It might be the next big proven solution, and I view it as a high priority to find out. Evidence Action’s management team is clearly excited about testing and proving programs, as demonstrated by their affiliation with Innovations for Poverty Action, GiveWell’s observations about their Deworm the World Program, and some of their own statements. So I plan to donate to Evidence Action, unrestricted, and allow them to use the funds for either of their programs—wherever they think it is needed most.
UNICEF is different from GiveWell’s recommended charities, which almost always focus on only a single intervention. GiveWell has expressed several concerns with mega-charities such as UNICEF, most notably that “they engage in a wide variety of activities, and we can’t get a concrete sense of (a) the specifics of the activities; (b) the organization-wide track record; (c) likely uses of additional funding.” I share that concern. In UNICEF’s case, 68% of their 2012 revenue was restricted, so it is not possible to look at their current activities and understand what they view as priorities or what they would do with additional unrestricted gifts. However, UNICEF has advantages over organizations that focus on a single program, and these should not be ignored. For example, there is value to having a lot of experience working in the developing world and having permanent in-country staff. Ironically, this value can be seen through AMF’s weaknesses; GiveWell removed AMF from its recommendations because of AMF’s inability to finalize a large net distribution, which GiveWell attributes to a communication style that put off potential partners and a lack of in-person communication. UNICEF is much less likely to have that type of problem.
In addition, UNICEF is a key implementer of several of the proven cost-effective interventions that GiveWell has had difficulty finding room for more funding for, including being the world’s largest purchaser of vaccines. UNICEF’s track record includes a number of programs that likely have higher impact than GiveWell’s top recommendations. I view that as a sign about UNICEF’s values and capabilities. If GiveWell is right about their room for more finding, my donation to UNICEF may not directly increase the scale of these programs. However, I believe there is significant value to giving them unrestricted support, which might be used to maintain their capabilities or expand other programs they view as priorities. Nevertheless, some of UNICEF’s programs are not in areas I view as priorities; I don’t know to what extent that is because of restricted grants versus UNICEF’s decisions. I wish I had more visibility into this, but that’s a risk I’m willing to take.
GiveDirectly is absent from the list of charities I’m donating to, though it is the favored charity of Good Ventures as well as most of GiveWell’s staff. GiveWell describes GiveDirectly’s intervention as having “an unusually low burden of proof.” Even GiveWell seems to state that GiveDirectly is probably less cost-effective than its other recommended charities. The suggested upside of GiveDirectly is that it will “improve the aid community’s understanding of, and interest in, cash transfer programs.” This seems to be a preference for strong evidence of impact over evidence of strong impact. I’m more excited about discovering, testing and promoting interventions with evidence of stronger impact.
After writing all that, it may be surprising that I’m still going to donate to GiveWell. I still believe they have a very strong team and are an extremely important resource for the giving community. Even when I don’t take their recommendations, the knowledge I gained from them has improved my giving. GiveWell Labs will loosen the requirements for evidence, which I look forward to. I also hope GiveWell will adjust its criteria for traditional charity recommendations to better balance the desires for high impact and robust evidence. Transparency and evidence is a good thing, but I believe GiveWell has emphasized these characteristics too much, to the exclusion of other important factors.
Eric Friedman is the author of Reinventing Philanthropy: A Framework for More Effective Giving. An excerpt is available at www.ReinventingPhilanthropy.com
We appreciate the thoughtful comments by Eric Friedman. At the same time, we feel that they misrepresent GiveWell's criteria and emphasis, and we feel it is important to clarify matters.
This post portrays GiveWell as placing strong weight on strength of evidence and as being highly averse to taking risks and chasing "upside," i.e., potential for greater impact. I don't think that's accurate, but I do think the kind of "upside" we're looking for is different from the kind Eric is looking for.
The "upside" Eric is seeking seems to be in the form of funding interventions with higher direct impact per dollar. It's worth noting that the variance here - within the range of interventions under consideration - is not particularly large. We estimate that deworming does 2-3x more direct good per dollar spent than cash transfers; given the low robustness of the calculation, I don't think it's straightforward to say which actually has higher expected direct good accomplished per dollar outright. Bednets appear have similar cost-effectiveness to deworming and stronger evidence, and my sense is that the best interventions carried out by UNICEF are in the same ballpark as well. The gains here would be around a factor of 3 if one believed our estimates are highly robust. I believe that the right number to use when accounting for low robustness is below a factor of 3, though how much below is a matter for debate.
The "upside" we're seeking, by contrast, is in the form of creating public goods and incentives that lead to better giving over time. Transparency is important because it allows more accurate cost-effectiveness analysis, assessment of room for more funding, and general learning that makes us better than evaluations. Generation of new evidence (something we believe GiveDirectly is more likely to provide than Evidence Action) is important for the same reasons. Rewarding charities that communicate clearly about their work and facilitate more learning is also important. I believe our emphasis on this sort of "upside" rather than on maximal (non-robustly) estimated cost-effectiveness has already led us to a much better position from which to be making informed giving decisions. I expect the vast majority of the good we and our donors do to lie well in the future, and so prioritizing factors relevant to learning and quality of information seems appropriate when the sacrifices in estimated-direct-impact terms are relatively small.
Another sort of "upside" we are interested in is that of scaling relatively small but high-potential organizations, especially innovative ones.
The contrast is particularly apparent in the case of GiveDirectly (strongly recommended by GiveWell, not supported by Eric) and UNICEF (supported by Eric, not recommended by GiveWell). Over the last year, GiveDirectly has attracted major attention in the media and in the development community. This attention may lead to dramatic rethinking of what aid should look like and how it should be evaluated. GiveDirectly is also pioneering unusual methods of aid delivery (mobile money) and evaluation/data collection (for example, using Google Earth to verify recipient eligibility). GiveDirectly has run an RCT of its own program and plans further experiments on both operational and programmatic components of cash transfers, which will produce new public goods. There are a lot of questions about GiveDirectly's future; we expect to follow along with it, learn a lot, and share our learning publicly. By contrast, it is probably not possible to learn much from following along with the impact of one's donations to UNICEF (it would not be possible even if GiveWell were to devote significant capacity to trying to do so), and my sense is that UNICEF's contribution to global public goods and rethinking aid is dramatically lower than GiveDirectly's when taking budget size into account. (Based on budget size and place in the aid community alone, it seems unlikely on its face that an $X donation to UNICEF has higher upside than an $X donation to GiveDirectly.) With all of this in mind, it doesn't strike me as correct to say that the case for GiveDirectly over UNICEF is all about evidence and confidence, or to say that UNICEF has greater upside.
The contrast between GiveDirectly and Evidence Action is also an important one to note. Evidence Action is affiliated with IPA, but it does not itself carry out rigorous studies. While I believe Evidence Action is an excellent organization and strongly support donations to it, I don't believe that donations to Evidence Action will lead to much more public knowledge about the effectiveness of chlorine dispensers or deworming, in the way that donations to GiveDirectly will lead to much more public knowledge about the impact of cash transfers.
I respect Eric's preference for high estimated cost-effectiveness over strong evidence; this is a judgment call that I disagree with but am not confident about. However, I think it is incorrect to say that GiveWell is neglecting upside entirely in its recommendations. I see GiveDirectly as the most innovative and high-upside of the charities discussed in this post.