Comment author: BenHoffman 17 February 2017 06:53:25AM 3 points [-]

Thanks for doing this!

I don't follow the argument that only one funder implies little RFMF:

I agree with gwern that it is concerning that the small number of Open Science orgs are mostly all funded by LJAF, despite the general awareness of the problems. For our purposes, this probably means that OS has little RFMF right now, because the opportunities are already filled by LJAF.

Could you say more about this?

I think Catherio's response to Gwern on that LessWrong thread is also worth reading. The gist is that the replication effort was already underway but unfunded when Arnold decided to fund it. So if you're looking for things to help with funding, you might want to ask around about other attempts within a high-value profession to coordinate around better standards.

Comment author: HoldenKarnofsky 17 February 2017 05:13:40AM 12 points [-]

Hi Ben,

Thanks for putting so much thought into this topic and sharing your feedback.

I'm going to discuss the reasoning behind the "splitting" recommendation that was made in 2015, as well as our current stance, and how they relate to your points. I'll start with the latter because I think that will make this comment easier to follow. I'll then address some more specific points and suggestions.

I'm not addressing recommendations addressed to GiveWell - I think it will make more sense for someone more involved in GiveWell to do that - though I will address both the 2015 and 2016 decisions about how much to recommend that Good Ventures support GiveWell's top charities, because I was closely involved in those decisions.

Current stance on Good Ventures support for GiveWell's top charities. As noted here, we (Open Phil) currently feel that the "last dollar" probably beats GiveWell's top charities according to our (extrapolated) values. We are quite uncertain of this view at this time and are hoping to do a more thorough investigation and writeup this year. We recommended $50 million to top charities for the 2016 giving season, for reasons laid out in that post and not discussed in the original post on this thread.

You seem to find our take on the "last dollar" a difficult-to-justify conclusion (or at least difficult to square with the fact that we are currently well under eventual peak giving, and not closing the gap via the actions you list under "symmetry"). You argue that the key issue here is the question of returns to scale, and say that we should regrant to larger organizations if we think returns are increasing, and smaller organizations if returns are decreasing. But I don't think the question "Are returns to scale increasing or decreasing?" is a particularly core question here (nor does it have a single general answer). Instead, our reason for thinking our "last dollar" can beat top charities and many other options is largely bound up in our model of ourselves as people who aspire to become "experts" in the domain of giving away large amounts of money effectively and according to the basic stance of effective altruism. I've written about my model of broad market efficiency in the past; I don't think it is trivial to "beat the market," but nor do I think it is prohibitively difficult, and I expect that we can do so in the long run. Another key part of the view is that there is more than one plausible worldview under which it looks (in the long run) quite tractable to spend essentially arbitrary amounts of money in a way that has better value for money than top charities (this is also discussed in the post on our current view ).

Previously, our best guess was different. We thought that the "last dollar" was worse than top charities - but not much worse, and with very low confidence. We fully funded things we thought were much better than the "last dollar" (including certain top charities grants) but not things we thought were relatively close when they also posed coordination issues. For this case, fully funding top charities would have had pros and cons relative to splitting: we think the dollars we spent would've done slightly more good, but the dollars spent by others would've done less good (and we think we have a good sense of the counterfactual for most of those dollars). We guessed that the latter outweighed the former.

I think that an important factor playing into both decisions, and a potentially key factor causing you and me to see things differently, pertains to conservatism. For the 2015 decision in particular, we didn't have much time to think carefully about these issues, and "fully funding" might be the kind of thing we couldn't easily walk back (we worried about a consistent dynamic in which our entering a cause led to other donors' immediately fleeing it). It's often the case that when we need to make high-stakes decisions without sufficient time or information, we err on the side of preserving option value and avoiding particularly bad outcomes (especially those that pose risks to GiveWell or Open Phil as an organization); this often leads to "hacky" actions that are knowably not ideal for any particular set of facts and values, if we had confidently sorted these facts and values out (but we haven't).

Responses to more specific points

"First, the adversarial framing here seems unnecessary. If the other player hasn’t started defecting in the iterated prisoner’s dilemma, why start?"

I don't think this is a case of "defecting" or "adversarial framing." We were trying to approximate the outcome we would've reached if we'd been able to have a friendly, open discussion and coordination with individual donors, which we couldn't.

"if you take into account the difference in scale between Good Ventures and other GiveWell donors, Good Ventures’s 'fair share' seems more likely to be in excess of 80%, than a 50-50 split."

We expected individual giving to grow over time, and thought that it would grow less if we had a policy of fully funding top charities. Calculating "fair share" based on current giving alone, as opposed to giving capacity construed more broadly and over a longer-term, would have created the kinds of problematic incentives we wrote that we were worried about. 50% is within range of what I'd guess would be a long-term fair share. Given that it is within range, 50% was chosen as a proportion that would (accurately) signal that we had chosen it fairly arbitrarily, in order to commit credibly to splitting, as mentioned in the post.

"This ethical objection doesn’t make sense. It implies that it’s unethical to cooperate on the iterated Prisoner’s Dilemma." The ethical objection was to being misleading, not to the game-theoretic aspects of the approach.

I don't follow your argument under "Influence via habituation vs track record." The reason there was "not enough money to cover the whole thing" was because we were unwilling to pay more than what we considered our fair share, due to the incentives it would create and the long-run implications for total positive impact. We were open about that. I also think that the "surface case" for low-engagement donors who didn't read our work was about as close to the truth as a surface case could be. (I would describe the "surface case" as something like: "If I give this money, then bednets will be delivered; if I do not, that will not happen." I do not believe that the majority of GiveWell donors - including very large donors - base their giving on Open Phil's opinions, or in many cases even know what Open Phil is.) I don't see how this situation implies any of your #1-#3, and I don't see how it is deceptive.

"Access via size" and "Independence via many funders" were not part of our reasoning.

(Continued in next comment)

Comment author: BenHoffman 17 February 2017 06:43:59AM 4 points [-]

Thanks for the detailed response! I wanted to quickly point out something you did here that I think is good practice, and wish more people did:

"Access via size" and "Independence via many funders" were not part of our reasoning.

Marking which parts of someone's argument you think are relevant and which you think aren't - and, relatedly, which branches of a disjunction you accept and which you reject - are an important part of how arguments can lead to shared models. A lot of people neglect this sort of thing, because it's not a clear way to score points for their side. You took care to address it here. Thanks.

(More to follow when I've had time to take this in.)

Comment author: Peter_Hurford  (EA Profile) 14 February 2017 12:05:47AM 1 point [-]

In addition to Will, this team consists of Emma Gray, Executive Assistant, and Roxanne Heston, Press Officer. This month they have been mainly focused on setting up the processes for the new team and helping to devise strategies to help build Will’s influence and connections.

I don't have any problem with this, but stating it so clearly and bluntly comes off to me as kind of cultish.

Comment author: BenHoffman 16 February 2017 09:16:45PM *  3 points [-]

I think this is admirable honesty about an influence-based strategy with a key person dependency. This makes it possible for people to look for ways to reduce risk, increase the upside, and point out possible downsides.

Comment author: Carl_Shulman 15 February 2017 06:01:06PM 2 points [-]

I don't think linear giving opportunities closely analogous to bednets will take $10BB without diminishing returns (although you might be able to beat that with R&D, advocacy, gene drives, and other leveraged strategies for a longer period). But I think this is a flawed argument.

If at current cost-effectiveness levels, top charities could scale up to solve that whole problem, then if we assume a cost of $5,000 per life saved, the whole thing would cost $50 billion/yr. That's more than Good Ventures has on hand - but it's not an order of magnitude more. It's not more than Good Ventures and its donors and the Gates foundation ($40 billion) and Warren Buffett's planned gifts to the Gates Foundation add up to - and all of those parties seem to be interested in this program area.

The original text strongly suggested a one-time cost, not a recurring annual cost. When you have diminishing returns in a single year (especially as programs are scaled up; BMGF has ramped up its spending over time), the fact that they don't spend everything in a firehose in a single year is far from shocking (note BMGF has spent a lot on US education too, it's not a pure global poverty focus although that is its main agenda).

GWWC's FAQ claims:

The Institute of Health Metrics and Evaluation estimated that between 2000 and 2014 the $73.6 billion spent on child health by donors (including both private and public) averted the death of 14 million infants and children. This is in addition to the $133 billion spent on child health by low- and middle-income country governments, which is estimated to have averted the deaths of 20 million children.

The annual figure for this is ~$14 billion (and not all spent where the evidence is best, including corruption, etc).

Gates Foundation spending is several billion dollars per year spread across a number of areas.

Total spending in these areas is not so large that a billion dollars a year is a drop in the bucket, and theses diseases have been massively checked or reduced (e.g. malaria, vaccinations, slowing HIV infections, smallpox eradication, salt iodization, etc).

And we haven't explicitly talked about possible leverage from R&D and advocacy in poverty.

Starting with a small area where you can show clear gains is not a new idea - it's the intuition behind Jeffrey Sachs's idea of millennium villages

Those were criticized at the time for spending so much on the same people, including less well-supported interventions and over diminishing returns, rather than doing more cost-effective interventions across a larger number of people. Local effectiveness of medical interventions is tested in clinical trials.

And remember that once you wipe out a communicable disease, it's much cheaper to keep it away; when's the last time people were getting smallpox?

Smallpox was a disease found only in humans with a highly effective vaccine. Such diseases are regularly locally extirpated, although getting universal coverage around the world to the last holdout regions (civil war, conspiracy theories about the vaccinations) can be very hard, as in polio eradication, and infectious diseases can quickly recolonize afterwards (malaria rebounded from the 60s failed eradication effort in places without continuing high quality prevention). But polio eradication is close and is a priority of e.g. Gates Foundation funding. It's also quite expensive, more than $10 billion so far. For harder to control diseases without vaccines like malaria, even moreso (and you couldn't just spend more in a big bang one year and be sure you haven't missed a spot).

Comment author: BenHoffman 15 February 2017 06:27:59PM *  1 point [-]

This seems like evidence for a combination of the second and third possibilities in the trilemma. Either GiveWell should expect to be able to point to empirical evidence of dramatic results soon (if not already), or it should expect to reach substantially diminishing returns, or both.

I agree that there are lots of practical reasons why you can't just firehose this stuff - that's part of the diminishing returns story!

I could imagine a scenario that slips in between 2 and 3, like you don't hit substantially diminishing returns on malaria until the last 1% of incidence, but is there reason to think that's the case?

Comment author: Linch 15 February 2017 12:39:21PM *  3 points [-]

The GiveWell Top Charities are part of the Open Philanthropy Project’s optimal philanthropic portfolio, when only direct impact is considered. There’s not enough money to cover the whole thing. These are highly unlikely to both be true. Global poverty cannot plausibly be an unfillable money pit at GiveWell’s current cost-per-life-saved numbers. At least one of these three things must be true:

GiveWell’s cost per life saved numbers are wrong and should be changed.

The top charities’ interventions will reach substantially diminishing returns long before they’ve managed to massively scale up.

A few billion dollars can totally wipe out major categories of disease in the developing world.

I don't think I understand the trilemma you presented here.

As a sanity check, under-5 mortality is about 6 million worldwide. Assuming that more than 2/3 is preventable (which I think is a reasonable assumption if you compare with developed world numbers on under-5 mortality), this means there are 4 million+ preventable deaths (and corresponding suffering) per year. At $10000 to prevent a death, this is already way more money than Open Phil has in a few months. At $3500 to prevent a death, this is still more money than Open Phil has in even a single year.

We would expect the numbers to also be much larger if we're not prioritizing just deaths, but also prevention of suffering.

Comment author: BenHoffman 15 February 2017 04:27:09PM *  0 points [-]

GBD 2015 estimates that communicable, maternal, neonatal, and nutritional deaths worldwide amount to about 10 million in 2015. And they are declining at a rate of about 20% per decade. If at current cost-effectiveness levels, top charities could scale up to solve that whole problem, then if we assume a cost of $5,000 per life saved, the whole thing would cost $50 billion/yr. That's more than Good Ventures has on hand - but it's not an order of magnitude more. It's not more than Good Ventures and its donors and the Gates foundation ($40 billion) and Warren Buffett's planned gifts to the Gates Foundation add up to - and all of those parties seem to be interested in this program area.

That's an extreme upper bound. It's not limited to the developing world, or to especially tractable problems. You almost certainly can't scale up that high at current costs - after all, the GiveWell top charities are supposed to be the ones pursuing the most important low-hanging fruit, tractable interventions for important but straightforward problems. But then, how high can you scale up at similar cost-effectiveness numbers? Can you do a single disease? For one continent? One region? One country? Now, we're getting to magnitudes that may fall well within Good Ventures's ability to fund the whole thing. (Starting with a small area where you can show clear gains is not a new idea - it's the intuition behind Jeffrey Sachs's idea of millennium villages.) And remember that once you wipe out a communicable disease, it's much cheaper to keep it away; when's the last time people were getting smallpox? Similarly, nutritional interventions such as food fortification tend to be permanent. There's a one-time cost, and then it's standard practice.

GBD 2015 estimates that there are only about 850,000 deaths due to neglected tropical diseases each year, worldwide. At $5,000 per life saved, that's about $4.2 billion to wipe out the whole category. Even less if you focus on one continent, or one region, or one country. To name one example, Haiti is a poor island with 0.1% of the world's population; can we wipe out neglected tropical diseases for $4.2 million there? $40 million?

Comment author: Carl_Shulman 15 February 2017 04:33:16AM *  6 points [-]

that seems like it would be the single most important consideration to mention in the 2015 post explaining splitting, and I am baffled as to why it was left out.

Ben, you have advocated just giving to the best thing at the margin, simply. Doing that while taking room for more funding into account automatically results in what you are calling 'defecting' here in this post (which I object to, since the game theoretic analogy is dubious, and you're using it in a highly morally charged way to criticize a general practice with respect to a single actor). That's a normal way of assessing donations in effective altruism, and common among strategic philanthropists.

The 'driving away donors' bit was repeatedly discussed, as was the routine occurrence of such issues in large-scale philanthropy (where foundations bargain with each other over shares of funding in areas of common interest).

Comment author: BenHoffman 15 February 2017 07:03:58AM 0 points [-]

I don't actually think it's defecting to take into account room for more funding. I do think it's defecting to try to control the behavior of other donors, who have more info about their opportunity cost than you do. Defecting is not always unjustified, but it's nice when we can find and maintain cooperate-cooperate equilibria.

I don't think it's unreasonable to describe major foundations as engaged in an iterated game where they display a combination of cooperative and uncooperative behavior to test each other's boundaries and guard their own in a moderately low-trust equilibrium. If you think there's something especially good about the EA way, it shouldn't be that surprising that large established charities sometimes engage in uncooperative behavior. I'm holding the Open Philanthropy Project and Good Ventures to a higher standard because they say they want to do better and I believe them.

My understanding is that GiveWell has mostly counted "leveraged" donations as costs towards their cost per life saved figures, rather than counting them as free money, and I think it's been right to do so. This seems like basically the same thing.

Comment author: Carl_Shulman 15 February 2017 04:26:13AM *  3 points [-]

Cross-posted from Ben's blog:

if Good Ventures committed to fully funding the GiveWell top charities, other donors might withdraw funding to fund the next-best thing by their values, confident that they’d be offset. A commitment to “splitting” would prevent this...

I have two main objections to this. First, the adversarial framing here seems unnecessary. If the other player hasn’t started defecting in the iterated prisoner’s dilemma, why start?

If GV fully funded the top charities, and others also funded them, then they would be overfunded by GV's lights. if A and B both like X (and have the same desired funding level for it), but have different second choices of Y and Z, the fully cooperative solution would not involve either A or B funding X alone.

[CoI notice: I consult for OpenPhil.]

Comment author: BenHoffman 15 February 2017 06:50:21AM *  1 point [-]

That's true! Fortunately, there are a few important mitigating factors:

  • This game proceeds in continuous time, so there's plenty of opportunity for donors to inform each other of their actions. For the GiveWell top charities, this often happens by reporting the donation to - or making it through - GiveWell.
  • As you've pointed out, excess donations - if they in fact turn out to be excess - can simply be funged against implicitly via lower room for more funding estimates in the following year.
  • A commitment to full funding doesn't have to take the form of initially giving them the whole amount - for instance, if the estimated funding gap is X, and GV would expect other donors to contribute amount X-Y if it weren't around, it can give Y, monitor other donations, and fill in gaps as they occur. It could even wait until after "giving season" to get more info.
Comment author: Carl_Shulman 15 February 2017 04:33:16AM *  6 points [-]

that seems like it would be the single most important consideration to mention in the 2015 post explaining splitting, and I am baffled as to why it was left out.

Ben, you have advocated just giving to the best thing at the margin, simply. Doing that while taking room for more funding into account automatically results in what you are calling 'defecting' here in this post (which I object to, since the game theoretic analogy is dubious, and you're using it in a highly morally charged way to criticize a general practice with respect to a single actor). That's a normal way of assessing donations in effective altruism, and common among strategic philanthropists.

The 'driving away donors' bit was repeatedly discussed, as was the routine occurrence of such issues in large-scale philanthropy (where foundations bargain with each other over shares of funding in areas of common interest).

Comment author: BenHoffman 15 February 2017 06:34:26AM *  0 points [-]

The prospect of driving away donors was discussed. Direct evidence of a reduction in donations wasn't, unless I missed something big. My impression is that donations from other sources were growing at the time and have continued to grow substantially from year to year.

Given that, I could maybe see the case for committing not to give more than the anticipated remainder assuming growth in other donations continued apace, as a credible threat against shirking, but 50-50 "splitting" massively undershoots that mark.

Comment author: RobBensinger 14 February 2017 04:40:53PM *  4 points [-]

Thanks for summarizing this, Ben!

First, the adversarial framing here seems unnecessary. If the other player hasn’t started defecting in the iterated prisoner’s dilemma, why start?

I might be getting this wrong, but my understanding is that a bunch of donors immediately started 'defecting' (= pulling out of funding the kinds of work GV is excited about) once they learned of GV's excitement for GW/OPP causes, on the assumption that GV would at some future point adopt a general policy of (unconditionally?) 'cooperating' (= fully funding everything to the extent it cares about those things).

I think GW/GV/OPP arrived at their decision in an environment where they saw a non-trivial number of donors preemptively 'defecting' either based on a misunderstanding of whether GW/GV/OPP was already 'cooperating' (= they didn't realize that GW/GV/OPP was funding less than the full amount it wanted funded), or based on the assumption that GW/GV/OPP was intending to do so later (and perhaps could even be induced to do if others withdrew their funding). If my understanding of this is right, then it both made the cooperative equilibrium seem less likely, and made it seem extra important for GW/GV/OPP to very loudly and clearly communicate their non-CooperateBot policy lest the misapprehension spread even further.

I think the difficulty of actually communicating en masse with smaller GW donors, much less having a real back-and-forth negotiation with them, played a very large role in GW/GV/OPP's decisions here, including their decision to choose an 'obviously arbitrary' split number like 50% rather than something more subtle.

It also assumes that people are taking the cost-per-life-saved numbers at face value, and if so, then GiveWell already thinks they’ve been misled.

I'm not sure I understand this point. Is this saying that if people are already misled to some extent, or in some respect, then it doesn't matter what related ways one's actions might confuse them?

(Disclaimer: I work for MIRI, which has received an Open Phil grant. As usual, the above is me speaking on my own behalf, not on MIRI's.)

Comment author: BenHoffman 14 February 2017 06:37:03PM *  1 point [-]

On cost per life saved numbers, I'm saying that "defend the state where these #s are true" is a silly goal for an organization that doesn't think anyone should have taken them literally in the first place. One of the many complicating factors for the cost per life saved numbers is that there are other inputs, some of which are complements to your donation, others of which are substitutes.

Comment author: RobBensinger 14 February 2017 04:40:53PM *  4 points [-]

Thanks for summarizing this, Ben!

First, the adversarial framing here seems unnecessary. If the other player hasn’t started defecting in the iterated prisoner’s dilemma, why start?

I might be getting this wrong, but my understanding is that a bunch of donors immediately started 'defecting' (= pulling out of funding the kinds of work GV is excited about) once they learned of GV's excitement for GW/OPP causes, on the assumption that GV would at some future point adopt a general policy of (unconditionally?) 'cooperating' (= fully funding everything to the extent it cares about those things).

I think GW/GV/OPP arrived at their decision in an environment where they saw a non-trivial number of donors preemptively 'defecting' either based on a misunderstanding of whether GW/GV/OPP was already 'cooperating' (= they didn't realize that GW/GV/OPP was funding less than the full amount it wanted funded), or based on the assumption that GW/GV/OPP was intending to do so later (and perhaps could even be induced to do if others withdrew their funding). If my understanding of this is right, then it both made the cooperative equilibrium seem less likely, and made it seem extra important for GW/GV/OPP to very loudly and clearly communicate their non-CooperateBot policy lest the misapprehension spread even further.

I think the difficulty of actually communicating en masse with smaller GW donors, much less having a real back-and-forth negotiation with them, played a very large role in GW/GV/OPP's decisions here, including their decision to choose an 'obviously arbitrary' split number like 50% rather than something more subtle.

It also assumes that people are taking the cost-per-life-saved numbers at face value, and if so, then GiveWell already thinks they’ve been misled.

I'm not sure I understand this point. Is this saying that if people are already misled to some extent, or in some respect, then it doesn't matter what related ways one's actions might confuse them?

(Disclaimer: I work for MIRI, which has received an Open Phil grant. As usual, the above is me speaking on my own behalf, not on MIRI's.)

Comment author: BenHoffman 14 February 2017 06:32:59PM *  1 point [-]

If a substantial share of other donors were already observed defecting, that seems like it would be the single most important consideration to mention in the 2015 post explaining splitting, and I am baffled as to why it was left out.

It seems like genuinely unfriendly behavior on the part of other donors and it would have been a public service at that time to call them out on this.

View more: Next