Comment author: Carl_Shulman 15 January 2017 08:12:21PM 4 points [-]

Looks like Tim Telleen-Lawton won, as the first ten digits of the beacon at noon PST were 0CF7565C0F=55689239567. Congratulations to Tim, and to all of the early adopters.

Comment author: Carl_Shulman 15 January 2017 08:09:04PM *  9 points [-]

Looks like Tim Telleen-Lawton won, as the first ten digits of the beacon at noon PST were 0CF7565C0F=55689239567. Congratulations to Tim, and to all of the early adopters.

Comment author: vipulnaik 12 January 2017 06:24:38AM 13 points [-]

The post does raise some valid concerns, though I don't agree with a lot of the framing. I don't think of it in terms of lying. I do, however, see that the existing incentive structure is significantly at odds with epistemic virtue and truth-seeking. It's remarkable that many EA orgs have held themselves to reasonably high standards despite not having strong incentives to do so.

In brief:

  • EA orgs' and communities' growth metrics are centered around numbers of people and quantity of money moved. These don't correlate much with epistemic virtue.
  • (more speculative) EA orgs' donors/supporters don't demand much epistemic virtue. The orgs tend to hold themselves to higher standards than their current donors.
  • (even more speculative; not much argument offered) Even long-run growth metrics don't correlate too well with epistemic virtue.
  • Quantifying (some aspects of) quality and virtue into metrics seems to me to have the best shot at changing the incentive structure here.

The incentive structure of the majority of EA-affiliated orgs has centered around growth metrics related to number of people (new pledge signups, number of donors, number of members), and money moved (both for charity evaluators and for movement-building orgs). These are the headline numbers they highlight in their self-evaluations and reports, and these are the numbers that people giving elevator pitches about the orgs use ("GiveWell moved more than $100 million in 2015" or "GWWC has (some number of hundreds of millions) in pledged money"). Some orgs have slightly different metrics, but still essentially ones that rely on changing the minds of large numbers of people: 80,000 Hours counts Impact-Adjusted Significant Plan Changes, and many animal welfare orgs count numbers of converts to veganism (or recruits to animal rights activism) through leafleting.

These incentives don't directly align with improved epistemic virtue! In many cases, they are close to orthogonal. In some cases, they are correlated but not as much as you might think (or hope!).

I believe the incentive alignment is strongest in cases where you are talking about moving moderate to large sums of money per donor in the present, for a reasonable number of donors (e.g., a few dozen donors giving hundreds of thousands of dollars). Donors who are donating those large sums of money are selected for being less naive (just by virtue of having made that much money) and the scale of donation makes it worth their while to demand high standards. I think this is related to GiveWell having relatively high epistemic standards (though causality is hard to judge).

With that said, the organizations I am aware of in the EA community hold themselves to much higher standards than (as far I can make out) their donor and supporter base seems to demand of them. My guess is that GiveWell could have been a LOT more sloppy with their reviews and still moved pretty similar amounts of money as long as they produced reviews that pattern-matched a well-researched review. (I've personally found their review quality improved very little from 2014 to 2015 and much more from 2015 to 2016; and yet I expect that the money moved jump from 2015 to 2016 will be less, or possibly even negative). I believe (with weaker confidence) that similar stuff is true for Animal Charity Evaluators in both directions (significantly increasing or decreasing review quality won't affect donations that much). And also for Giving What We Can: the amount of pledged money doesn't correlate that well with the quality or state of their in-house research.

The story I want to believe, and that I think others also want to believe, is some version of a just-world story: in the long run epistemic virtue ~ success. Something like "Sure, in the short run, taking epistemic shortcuts and bending the truth leads to more growth, but in the long run it comes back to bite you." I think there's some truth to this story: epistemic virtue and long-run growth metrics probably correlate better than epistemic virtue and short-run growth metrics. But the correlation is still far from perfect.

My best guess is that unless we can get a better handle on epistemic virtue and quantify quality in some meaningful way, the incentive structure problem will remain.

Comment author: Carl_Shulman 12 January 2017 07:52:49AM 16 points [-]

One bit of progress on this front is Open Phil and GiveWell starting to make public and private predictions related to grants to improve their forecasting about outcomes, and create track records around that.

There is significant room for other EA organizations to adopt this practice in their own areas (and apply it more broadly, e.g. regarding future evaluations of their strategy, etc).

I believe the incentive alignment is strongest in cases where you are talking about moving moderate to large sums of money per donor in the present, for a reasonable number of donors (e.g., a few dozen donors giving hundreds of thousands of dollars). Donors who are donating those large sums of money are selected for being less naive (just by virtue of having made that much money) and the scale of donation makes it worth their while to demand high standards. I think this is related to GiveWell having relatively high epistemic standards (though causality is hard to judge).

This is part of my thinking behind promoting donor lotteries: by increasing the effective size of donors, it lets them more carefully evaluate organizations and opportunities, providing better incentives and resistance to exploitation by things that look good on first glance but don't hold up on close and extended inspection (they can also share their findings with the broader community).

The story I want to believe, and that I think others also want to believe, is some version of a just-world story: in the long run epistemic virtue ~ success. Something like "Sure, in the short run, taking epistemic shortcuts and bending the truth leads to more growth, but in the long run it comes back to bite you." I think there's some truth to this story: epistemic virtue and long-run growth metrics probably correlate better than epistemic virtue and short-run growth metrics. But the correlation is still far from perfect.

The correlation gets better when you consider total impact and not just growth.

In response to High Impact Science
Comment author: LaurenMcG  (EA Profile) 11 January 2017 06:22:18PM 0 points [-]

Amazing article! Are there any resources available to help identify, prioritise and facilitate opportunities for high-impact science? I'm currently researching cause prioritisation of, and within, biotechnology - especially it's positive applications. Any ideas would be greatly appreciated :)

In response to comment by LaurenMcG  (EA Profile) on High Impact Science
Comment author: Carl_Shulman 11 January 2017 07:42:52PM 1 point [-]

You might be interested in reading the Open Philanthropy Project's published science investigations, or 80000 Hours' write-ups on biomedical research, but I'm afraid they may not be developed enough for your purposes.

Comment author: Mac- 31 December 2016 02:57:00PM 1 point [-]

I think this is a very good idea. Unfortunately, I don't really know any of you, and I don't think it's worth the time to thoroughly research your reputations and characters, so I'm not going to contribute.

However, I would be interested in a registered charitable organization whose sole purpose is to run a donation lottery annually. In fact, I would donate to the operations of such a charity if the necessary safeguards and/or reputation were in place. Seems like an easy "bolt-on" project for GiveWell, no?

If anyone else would like to see a permanent donor lottery from GiveWell, let me know how much you're willing to contribute to start it (via private message if you prefer). I'll total the amounts in a few weeks and present to GiveWell. Maybe it will pique their interest.

Comment author: Carl_Shulman 11 January 2017 03:30:37AM *  1 point [-]

Thanks Mac, I agree that's the general direction this should go given success with this initial trial (and the lessons from actually doing it), and hope there is a good option for you in future (the current lottery is closed for the last few days pre-draw). That organization may or may not be GiveWell, although they are certainly aware of it.

6 of the 18 participants are GiveWell employees (including Open Philanthropy Project staff), one is a former GiveWell employee, and one is an advisor to the Open Philanthropy Project (as are Paul Christiano and myself).

Holden Karnofsky wrote favorably about it in the GiveWell blog post on staff members' personal donations, along with Tim, Ajeya, and Helen (the latter 3 participated).

We also (with the consent of the participants) rescaled the win probabilities and payout recommendation to guarantee a winner (who can then make a second bet to get from $45,600 to $100,000), so as to ensure we can learn from that process too before trying a bigger institutional version. See Paul's recent follow-up post.

So I'm hopeful something along the lines you suggest can be managed in future (although this round is closed now).

Comment author: zdgroff 02 January 2017 08:12:46PM 1 point [-]

I think Facebook and email dominate a large portion of nearly everyone's internet usage. I've seen other groups try to establish separate sites or discussion boards outside of Facebook and it never seems to work (EA Forum is probably successful relatively) because people just have so much social investment on Facebook.

Comment author: Carl_Shulman 02 January 2017 10:05:08PM *  1 point [-]

The EA Facebook group also has had slower growth in traffic than GWWC pledges or GW donations.

Comment author: vipulnaik 31 December 2016 06:48:05PM 0 points [-]

Some people in the effective altruist community have argued that small donors should accept that they will use marginal charitable dollars less efficiently than large actors such as Open Phil, for lack of time, skill, and scale to find and choose between charitable opportunities. Sometimes this is phrased as advice that small donors follow GiveWell's recommendations, while Open Phil pursues other causes and strategies such as scientific research and policy.

The argument that I have heard is a little different. It is that the entry of big players like Open Phil has made it harder to have the old level of marginal impact with one's donation.

Basically:

Marginal impact of one's donation now that Open Phil is plucking a lot of low-hanging fruit < Marginal impact of one's donation a few years ago ... (1)

Whereas the claim that you are critiquing is:

Marginal impact of one's donation < Marginal impact of Open Phil's donation ... (2)

Why does (1) matter? Some donors have fixed charity budgets, i.e., they wish to donate a certain amount every year to charity. For them, then, the challenge is just to find the best use of money, so even if marginal impacts are down across the board, it doesn't matter much because all the matters is relative impact.

For other donors and potential donors, charitable donations compete with other uses of money. Therefore, whether or not one donates to charity, and how much one donates to charity, would depend on how large the marginal impact is. If the entry of players like Open Phil has reduced the marginal impact achievable, then that's good reason to donate less.

So I feel that the argument you are attacking isn't the actually correct one to attack. Though you do address (1) a bit in the post, I think it would have made more sense to make it the main focus.

Comment author: Carl_Shulman 31 December 2016 07:05:30PM *  4 points [-]

So I feel that the argument you are attacking isn't the actually correct one to attack. Though you do address (1) a bit in the post, I think it would have made more sense to make it the main focus.

This sounds like a fine topic for another post to me.

Re diminishing marginal returns, note that some of the change is Open Phil finding areas that it tentatively guesses may be higher return than GiveWell top charities. That affords room for increase in expected returns via research and allocation (I discussed the balance of improved knowledge with diminishing returns above).

For donors who were already trying to pursue low-hanging fruit in areas OpenPhil hadn't reached, advantage has declined some, but not collapsed due to the continued presence of scale diseconomies discussed in the post (such as organizational independence and nonmonetary costs).

Comment author: Owen_Cotton-Barratt 31 December 2016 05:18:28PM 5 points [-]

Thanks for such a thorough exploration of the advantages of scaling up, and why small donors may be able to beat larger institutions at the margin. I'd previously (prior to the other thread) thought that there was typically not that much to gain (or lose) from entering a lottery, but I'm now persuaded that it's probably a good idea for many small donors.

I still see a few reasons one might prefer not to commit to a lottery:

1) If you see significant benefit to the community of more people seriously thinking through donation decisions, you might prefer to reserve enough of your donation to be allocated personally that you will take the process seriously (even if you give something else to a donor lottery). Jacob Steinhardt discusses this in his recent donation post. I'm sympathetic to this for people who actively want to take some time to think through where to give (but I don't think that's everyone).

2) If you prefer giving now over giving later, you may wish to make commitments about future donations to help charities scale up faster. This is much harder to do with donor lotteries. If you trusted other lottery entrants enough you could all commit to donating to it in the future, with the ability to make commitments about next year's allocation of funds being randomised today. But that's a much higher bar of trust than the current lottery requires. Alternatively you could borrow money to donate more (via the lottery) today. If you think that there are significant advantages to the lottery and to giving earlier, this strategy might be correct, even if borrowing to give to a particular charity is often beaten by making commitments about future donations. But if you think you're only getting a small edge from entering the lottery, this might be smaller than the benefit of being able to make commitments, and so not worthwhile.

3) If you think you might be in a good position to recognise small giving opportunities which are clearly above the bar for the community as a whole to fund, it could make sense for you to reserve some funds to let you fill these gaps in a low-friction manner. I think this is most likely to be the case for people who are doing direct work in high-priority areas. Taking such opportunities directly can avoid having to pull the attention of large or medium-sized funders. This is similar to the approach of delegating to another small donor, where the small donor is future-you.

Comment author: Carl_Shulman 31 December 2016 06:37:54PM 2 points [-]

Thank you for the thoughtful comments Owen.

If you see significant benefit to the community of more people seriously thinking through donation decisions, you might prefer to reserve enough of your donation to be allocated personally that you will take the process seriously (even if you give something else to a donor lottery). Jacob Steinhardt discusses this in his recent donation post

I agree. In the donor lottery FAQ and announcement post, this is discussed. Ctrl-f for 'practice people get thinking about charity.' I raise the possibility of setting some money aside for practice and symbolic donations. Jacob using his particular skills to identify CS opportunities (which he can also recommend to larger donors) makes particular sense. And he is putting 40% into the donor lottery too, which is your #3.

If you prefer [giving now over giving later, you may wish to make commitments about future donations to help charities scale up faster. This is much harder to do with donor lotteries

I agree with this in cases where the universe of donors available is or will be small, and that donor lotteries don't address committing/borrowing on future donations. On the other hand, strong small growth startups should be a limited portion of the charitable universe, and could tend to get extra funds from more informed donors (who could structure their grants over one or more years).

If you think you might be in a good position to recognise small giving opportunities which are clearly above the bar for the community as a whole to fund, it could make sense for you to reserve some funds to let you fill these gaps in a low-friction manner.

This is true if you expect such opportunities to be smaller than your donation/DAF saving when they arise. If they are often larger, then you might do better to team up to form a low-friction risk pool with others, at least when the donations would be to charities that can receive money from a DAF. There is also a class of opportunities that are not tax deductible, and can't be funded through a DAF (leading the Zuckerberg-Chan Initiative to set up as an LLC so that they can also, e.g. invest in companies).

Comment author: Owen_Cotton-Barratt 31 December 2016 05:50:49PM 2 points [-]

I think "in expectation" is meant to mean that they can access a probability of having large donation size and time investment. You might say "stochastically".

Comment author: Carl_Shulman 31 December 2016 06:14:31PM 0 points [-]

Owen is right.

19

Risk-neutral donors should plan to make bets at the margin at least as well as giga-donors in expectation

[Disclosures and disclaimers: I do work for the Future of Humanity Institute, and significant consulting for Open Phil. I have previously worked at MIRI, and consulted for 80,000 Hours/CEA. I have advised the EA Giving Group DAF regarding donations, and suggested to Paul Christiano that he facilitate the donor lottery... Read More

View more: Next