Comment author: Jon_Behar 26 May 2017 08:58:13PM 0 points [-]

Fair point about outliers driving the mean. Does suggest that a cost-effectiveness estimate should just try to quantify those outliers directly instead of going through a translation.
E.g. if "some of the 10s are likely to donate millions to charity within the next few years", just estimate the value of that rather than assuming that giving will on average equal 10x GWWC's estimate for the value of a pledge.

Comment author: Ben_Todd 27 May 2017 03:19:31AM 1 point [-]

Does suggest that a cost-effectiveness estimate should just try to quantify those outliers directly instead of going through a translation.

Yes, that's the main way I think about our impact. But I think you can also justify it on the basis of getting lots of people make moderate changes, so I think it's useful to consider both approaches.

Comment author: ThomasSittler 23 May 2017 10:37:49AM 1 point [-]

Hi Rohin, thanks for the comment! :) My hunch is also that 80,000 Hours and most organisations have diminishing marginal cost-effectiveness. As far as I know from our conversations, on balance this is Sindy's view too.

The problem with qualitative considerations is that while they are in some sense useful standing on their own, they are very difficult to aggregate into a final decision in a principled way.

Modelling the potential for growth quantitatively would be good. Do you have a suggestion for doing so? The counterfactuals are hard.

Comment author: Ben_Todd 26 May 2017 04:44:46AM 0 points [-]

My hunch is also that 80,000 Hours and most organisations have diminishing marginal cost-effectiveness. As far as I know from our conversations, on balance this is Sindy's view too.

You need to be very careful about what margin and output you're talking about.

As I discuss in my long comment above, I think it's unclear whether our annual ratio of cost per plan change will go up or down, and I think there's a good chance it continues to drop, as it has the last 4 years.

On the other hand, if you're talking about total value created per dollar (including all forms of value), then that seems like it's more likely to be going down. It seems intuitive that our earliest supporters who made 80k possible had more impact than supporters today.

Though even that's not clear. You could get increasing returns due to economies of scale or tipping point effects and so on.

Comment author: rohinmshah  (EA Profile) 14 May 2017 12:56:54AM 5 points [-]

Attracting more experienced staff with higher salary and nicer office: more experienced staff are more productive which would increase the average cost-effectiveness above the current level, so the marginal must be greater than the current average.

Wait, what? The costs are also increasing, it's definitely possible for marginal cost effectiveness to be lower than the current average. In fact, I would strongly predict it's lower -- if there's an opportunity to get better marginal cost effectiveness than average cost effectiveness, that begs the question of why you don't just cut funding from some of your less effective activities and repurpose it for this opportunity.

Given the importance of such considerations and the difficulty of modelling them quantitatively, to holistically evaluate an organization, especially a young one, there is an argument for using a qualitative approach and “cluster thinking”, in addition to a quantitative approach and “sequential thinking.”

Please do, I think an analysis of the potential for growth (qualitative or quantitative) would significantly improve this post, since that consideration could easily swamp all others.

Comment author: Ben_Todd 26 May 2017 04:40:55AM 0 points [-]

Wait, what? The costs are also increasing, it's definitely possible for marginal cost effectiveness to be lower than the current average.

Yes, agree with this. Like I say in the long comment above, I think that giving money to us right now probably has diminishing returns because we already made our funding targets for this year.

Comment author: Peter_Hurford  (EA Profile) 21 May 2017 11:17:47PM 1 point [-]

I think you should add more uncertainty to your model around the value of an 80K career change (in both directions). While 1 impact-adjusted change is approximately the value of a GWWC pledge, that doesn't mean it is equal in both mean and standard deviation as your model suggests, since the plan changes involve a wide variety of different possibilities.

It might be good to work with 80K to get some more detail about the kinds of career changes that are being made and try to model the types of career changes separately. Certainly, some people do take the GWWC pledge, and that is a change that is straightforwardly comparable with the value of the GWWC pledge (minus concerns about the counterfactual share of 80K), but other people make much higher-risk higher-reward career changes, especially in the 10x category.

Speaking just for me, in my personal view looking at a few examples of the 80K 10x category, I've found them to be highly variable (including some changes that I'd personally judge as less valuable than the GWWC pledge)... while this certainly is not a systematic analysis on my part, it would suggest your model should include more uncertainty than it currently does.

Lastly, I think your model right now assumes 80K has 100% responsibility for all their career changes. Maybe this is completely fine because 80K already weights their reported career change numbers for counterfactuality? Or maybe there's some other good reason to not take this into account? I admit there's a good chance I'm missing something here, but it would be nice to see it addressed more specifically.

Comment author: Ben_Todd 26 May 2017 04:39:27AM 1 point [-]

Lastly, I think your model right now assumes 80K has 100% responsibility for all their career changes. Maybe this is completely fine because 80K already weights their reported career change numbers for counterfactuality? Or maybe there's some other good reason to not take this into account? I admit there's a good chance I'm missing something here, but it would be nice to see it addressed more specifically.

I don't think that's true, because the GWWC pledge value figures have been counterfactually adjusted, and because we don't count all of the people we've influenced to take the GWWC pledge.

More discussion here: https://80000hours.org/2016/12/has-80000-hours-justified-its-costs/#giving-what-we-can-pledges

While 1 impact-adjusted change is approximately the value of a GWWC pledge, that doesn't mean it is equal in both mean and standard deviation as your model suggests, since the plan changes involve a wide variety of different possibilities.

Agree with that - the standard deviation should be larger.

Comment author: Jon_Behar 26 May 2017 12:07:32AM 2 points [-]

Thanks for sharing this analysis (and the broader project)!

Given the lengthy section on model limitations, I would have liked to have seen a discussion of sensitivity to assumptions. The one that stood out to me was the estimate for the value of a GWWC Pledge, which serves as a basis for all your calcs. While it certainly seems reasonable to use their estimate as a baseline, there’s inherently a lot of uncertainty in estimating a multi-decade donation stream and adjusting for counter-factuals, time discounting, and attrition.

FWIW, I’m pretty dubious about the treatment of plan changes scored 10. The model implies each of those plan changes is worth >$500k (again, adjusted for counterfactuals, time discounting, and attrition), which is an extremely high hurdle to meet. If a university student tells me they're going to "become a major advocate of effective causes" (sufficient for a score of 10), I wouldn't think that has the same expected value as a half million dollars given to AMF today.

Comment author: Ben_Todd 26 May 2017 04:36:39AM 2 points [-]

Hi Jon,

I would have liked to have seen a discussion of sensitivity to assumptions.

I agree - I think, however, you can justify the cost-effectiveness of 80k in multiple, semi-independent ways, which help to make the argument more robust:

https://80000hours.org/2016/12/has-80000-hours-justified-its-costs/

FWIW, I’m pretty dubious about the treatment of plan changes scored 10. The model implies each of those plan changes is worth >$500k...If a university student tells me they're going to "become a major advocate of effective causes" (sufficient for a score of 10), I wouldn't think that has the same expected value as a half million dollars given to AMF today.

Yes, we only weigh them at 10, rather than 40. However, here are some reasons the 500k figure might not be out of the question.

First, we care about the mean value, not the median or threshold. Although some of the 10s will probably have less impact than 500k to AMF now, some of them could have far more. For instance, there's reason to think GPP might have had impact equivalent to over $100m given to AMF. https://80000hours.org/2016/12/has-80000-hours-justified-its-costs/#global-priorities-project

You only need a small number of outliers to pull up the mean a great deal.

Less extremely, some of the 10s are likely to donate millions to charity within the next few years.

Second, most of the 10s are focused on xrisk and meta-charity. Personally, I think efforts in these causes are likely at least 5-fold more cost-effective than AMF, so they'd only need to donate a 100k to have as much impact as 500k to AMF.

Comment author: Ben_Todd 26 May 2017 04:29:17AM *  1 point [-]

Hi there,

Thanks for writing this. A couple of quick comments (these are not thoroughly checked - our annual reviews are the more reliable source of information):

How should we think about all these things they could do with additional funding now?

Given that we made the higher end of our funding targets, I'd guess that giving us money right now has diminishing returns compared to those we received earlier in the year. However, they are not super diminishing. First, they give us the option to grow faster. Second, if we don't take that option, then the worst case scenario is that we raise less money next funding round. This means you funge with our marginal donor in early 2018 (which might well be Open Phil), while also saving us time, and giving us greater financial strength in the meantime, which helps to attract staff.

Will our returns diminish from 2016 to 2017? That's less clear.

If you're looking at the ratio of plan changes to costs each year, as you do in your model, then there's a good chance the ratio goes down in 2017. Past investments will pay off, we learn how to be more efficient, and we get economies of scale. More discussion here: https://80000hours.org/2016/12/has-80000-hours-justified-its-costs/#whats-the-marginal-cost-per-plan-change

On the other hand, if we invest a lot in long-term growth, then the short-term ratio will go up.

This shows some of the limitation looking at the ratio of costs to plan changes each year, which we discuss more here: https://80000hours.org/2015/11/take-the-growth-approach-to-evaluating-startup-non-profits-not-the-marginal-approach/

If you're reading this and trying to evaluate 80,000 Hours, then I'd encourage you to consider other questions, which are glossed over in this analysis, but similarly, or more important, such as:

1) Is the EA community more talent constrained than funding constrained?

2) Will 80k continue to grow rapidly?

3) How pressing a problem are poor career choice and promoting EA?

4) How effective is AMF vs other EA causes? (80k isn't especially focused on global poverty)

5) Is 80k a well-run organisation with a good team?

You can see more of our thoughts on how to analyse a charity here: https://80000hours.org/articles/best-charity/

Comment author: PeterSinger 13 May 2017 11:53:35PM 5 points [-]

Why is the choice not directly comparable? If it were possible to offer a blind person a choice between being able to see, or having a guide dog, would it be so difficult for the blind person to choose?

Still, if you can suggest better comparisons that make the same point, I'll be happy to use them.

Comment author: Ben_Todd 15 May 2017 04:13:51AM *  9 points [-]

Hi Peter,

Some examples that might be useful:

1) Differences in income

A US college graduate earns about 100x more than GiveDirectly recipients, suggesting money can go far further with GiveDirectly. (100x further if utility ~log-income.) https://80000hours.org/career-guide/anyone-make-a-difference/

2) The cost to save a life

GiveWell now says $7500 for a death prevented by malaria nets (plus many other benefits) Rich country governments, however, are often willing to pay over $1m to save a life of one of their citizens, a factor of 130+ difference. https://80000hours.org/career-guide/world-problems/#global-health-a-problem-where-you-could-really-make-progress

3) Cost per QALY

It still seems possible to save QALYs for a few hundred dollars in the developing world, whereas the UK's NHS is willing to fund most things that save a QALY for under £20,000, and some that are over £30,000, which is again a factor of 100 difference.

So I still think a factor of 100x difference is defensible, though if you also take into account Brian's point below, then it might be reduced to, say, a factor of 30, though that's basically just a guess, and it could go the other way too. More on this: http://reflectivedisequilibrium.blogspot.com/2014/01/what-portion-of-boost-to-global-gdp.html

Comment author: the_jaded_one 26 April 2017 05:27:42PM *  0 points [-]

it would indeed be harder to live in the west on $2/day, because the low-quality goods that the global poor use are not available to buy. I think the relevant comparison is more like "if there were lots of people living on $2/day in the west, what quality of living would you get?". It's artificial to imagine one person living in extreme poverty without a market and community around them.

OK, so maybe appeals to donate money based on factors of 100 wealth difference should be limited to people who actually have a third-world price/quality market for (food, accommodation, shelter) available to them. Hmmmm OK that would be no-one at all.

Then we come to this:

You could, of course, doubt the existing estimates. My general policy is to go with the expert view when it comes to issues that have been thoroughly researched,

...

The PPP adjustments are meant as a hypothetical "what you could buy on $2 per day if the same goods were sold in stores, or if there were lots of other poor people in the country".

So they've thoroughly researched a question which is completely different than the one I care about, which is what I can actually buy and do.

As an inhabitant of a rich country, you get to consume lots of extra public goods that aren't fully included in the post-tax income figures in these data-sets e.g. safety, clean air, beautiful buildings, being surrounded by lots of educated people.

But these things are mostly not worth the massive amount of tax money I have to pay. And that's partly because that tax money is not being spent on me, (I have looked at government spending and the part of the pie that is spent on "things that childless healthy 30 year-olds want" is extremely small.), partly because taxation is progressive so punishes people who earn well, and partly because others in the west have different preferences about how much to spend reducing various risks (such as the risk of a $50,000 car being damaged in a collision with my $1000 old banger).

I would contend that I am not (on $60k) 100 times richer than the average Indian, at least not in the same way that someone on $6M is 100 times richer than me; the way that they really can buy my entire life 90 times over and still be way better off than me.

Anyway, thanks for responding to me, best regards, Jaded.

Comment author: Ben_Todd 27 April 2017 05:37:21AM 1 point [-]

OK, so maybe appeals to donate money based on factors of 100 wealth difference should be limited to people who actually have a third-world price/quality market for (food, accommodation, shelter) available to them. Hmmmm OK that would be no-one at all.

I agree that makes sense given one interpretation of the claim. But that definition also has some odd implications. Why does the actual option need to be available to you, even if you're never going to take it?

If a shanty town opens down the road from me, giving me the option to live like the global poor, I become richer relative to my neighbors, but I don't become richer in absolute terms. The reason the super cheap goods the global poor buy don't exist in the West is because no-one wants them. Even if a shanty town opened, I'd buy the same stuff as before, so my quality of life would be exactly the same. Your definition, however, would say I've become ~10x richer, which seems odd.

I think both senses of the term are relevant and interesting. Personally, I find the sense used by the World Bank better at capturing what I intuitively think about when comparing living standards and income. But it's useful to consider both.

You also haven't shown that the differences would amount to a factor of 10. Even though the exact goods the global poor would use are not available in the West, there are still very cheap goods available (as in Will's comment).

But these things are mostly not worth the massive amount of tax money I have to pay.

That's a good point - you're likely a net contributor of taxes right now. But many of things I mentioned aren't a result of taxes. There are lots of public goods produced by being around lots of other educated, wealthy people that you benefit from but aren't captured in the income figures, such as lower crime, more beautiful buildings, more opportunities to talk to people like that, lack of sewage on the street and so on.

Moreover, you're going to get some of those taxes back in the future (and you've benefited from taxes when younger). I think the lifecycle comparison is more relevant.

Over your life, you're probably not net-losing more than 10-20% of your income, so it's not a big factor in the comparison. We're looking for a 10x difference, not a 10% difference.

You can also reclaim tax on donations, so if the message if to donate more, arguably we should use pre-tax income instead.

Comment author: Kerry_Vaughan 23 April 2017 07:32:52PM 1 point [-]

I'm not sure that's true. There are a lot of venture funds in the Valley but that doesn't mean it's easy to get any venture fund to give you money.

I don't have the precise statistics handy, but my understanding is that VC returns are very good for a small number of firms and break-even or negative for most VC firms. If that's the case, it suggests that as more VCs enter the market, more bad companies are getting funded.

Comment author: Ben_Todd 24 April 2017 03:28:44AM *  3 points [-]

This is a huge digression, but:

I'm not sure it's obvious that current VCs fund all the potentially top companies. If you look into the history of many of the biggest wins, many of them nearly failed multiple times and could have easily shut down if a key funder didn't exist (e.g. Airbnb and YC).

I think a better approximation is an efficient market, in which the risk-adjusted returns of VC at the margin are equal to the market. This means that the probability of funding a winner for a marginal VC is whatever it would take for their returns to equal the market.

Then also becoming a VC, to a first order, has no effect on the cost of capital (which is fixed to the market), so no effect on the number of startups formed. So you're right that additional VCs aren't helpful, but it's for a different reason.

To a second order, there probably are benefits, depending on how skilled you are. The market for startups doesn't seem very efficient and requires specialised knowledge to access. If you develop the VC skill-set, you can reduce transaction costs and make the market for startups more efficient, which enables more to be created.

Moreover, the more money that gets invested rather than consumed, the lower the cost of capital in the economy, which lets more companies get created.

The second order benefits probably diminish as more skilled VCs enter, so that's another sense in which extra VCs are less useful than those we already have.

Comment author: the_jaded_one 06 April 2017 05:37:09PM *  2 points [-]

these bottom lines remain in every estimate of the global income distribution I’ve seen so far... Many people in the world live in serious absolute poverty, surviving on as little as one hundredth the income of the upper-middle class in the US.

But is this bottom line really approximately true?

A salary of $70,000 could be considered upper-middle-class. 1/100th of $70,000 is $700.

According to the chart, that is slightly greater than the income of the median Indian, adjusted for PPP.

Since these figures have been adjusted, that should mean that $700 in Western Europe or the US will afford you the same quality of life as the median Indian person, without you getting any additional resources such as extra meals from sympathetic passers-by or free accommodation in a shelter (because otherwise, to be 100 times richer you would have to have 100 units per day of these additional resources - i.e. $70,000 plus 100 meals/day plus owning low-quality accommodation for 100 people).

However, $700/year (= $1.91/day, =€1.80/day, =£1.53 /day) (without gifts or handouts) is not a sufficient amount of money to be alive in the west. You would be homeless. You would starve to death. In many places, you would die of exposure in the winter without shelter. Clearly, the median person in India is better off than a dead person.

A realistic minimum amount of money to not die in the west is probably $2000-$5000/year, again without gifts or handouts, implying that to be 100 times richer than the average Indian, you have to be earning at least $200,000-$500,000 net of tax (or at least net of that portion of tax which isn't spent on things that benefit you - which at that level is almost all of it, unless you are somehow getting huge amounts of government money spent on you in particular).

The reality is that a PPP conversion factor is trying to represent a nonlinear mapping with a single straight line, and it fails badly at the extremes. But the extremes are exactly where one is getting this (misleading) factor of 100 from.

Comment author: Ben_Todd 12 April 2017 04:19:02AM *  4 points [-]

Hey, a few comments:

But is this bottom line really approximately true?

Rob is saying “in every estimate of the global income distribution I’ve seen so far”, there has been a 100:1 ratio, which is true because this is what's shown by all the official data.

You could, of course, doubt the existing estimates. My general policy is to go with the expert view when it comes to issues that have been thoroughly researched, unless I've looked into it a lot myself. At 80,000 Hours, we don't see ourselves as experts on measuring global income, so instead go with the World Bank, Milanovic and others. Moreover, to our knowledge, the objections raised here are all well understood by the experts on the topic, and have already been factored into the analyses.

That said, here a few comments to show why the Milanovic etc. estimates are not obviously wrong. First, I'll state the problem, then consider the arguments.

I'm claiming that US upper middle class = $100k+ for the reason above. The World Bank estimated 800m people live under $1.9 per day in their 2015 figures, or $600 per year. In reality, many of them will live well below that level. So there's probably hundreds of millions living under $300 per year. This means there's over a factor of 300 difference between "upper middle class" and "large numbers of the global poor"

So, for the claim to be strictly wrong, the consumption of the poor have to be relatively underestimated by a factor of 3. Given the difficulties in making these estimates, this doesn't seem out of the question, but is not obvious.

Moreover, for it to be wrong in a way that becomes decision-relevant, you'd need the understatement to be more like a factor of 10. Even then, it would still be true that upper middle class earn 30x what the global poor earn, so they'd still be able to benefit others at little cost of themselves, and have a disproportionate influence on the world. But, it would be less pressing than a 300x difference.

Here are a couple of reasons why it's not obvious the world bank etc is off by a factor of 10.

It seems the main argument is that you'd die with under $2/day in the west and no hand-outs, so quality of life is worse than $2 in the developing world.

Will gives one response to that in another comment. You wouldn't actually die.

Another point is that it would indeed be harder to live in the west on $2/day, because the low-quality goods that the global poor use are not available to buy. I think the relevant comparison is more like "if there were lots of people living on $2/day in the west, what quality of living would you get?". It's artificial to imagine one person living in extreme poverty without a market and community around them. The PPP adjustments are meant as a hypothetical "what you could buy on $2 per day if the same goods were sold in stores, or if there were lots of other poor people in the country". (Though of course this is one reason why the comparisons are difficult conceptually.)

You've argued that the incomes of the poor are understated, but I think the incomes of the rich are also understated. As an inhabitant of a rich country, you get to consume lots of extra public goods that aren't fully included in the post-tax income figures in these data-sets e.g. safety, clean air, beautiful buildings, being surrounded by lots of educated people. You wouldn't get as many of these in a poor community in a poor country, so this is a way in which the global poor are relatively even worse off than the income comparisons suggest.

Finally, you mention cost of living in another comment as being relevant. Our audience lives in cities where cost of living is higher, making them relatively poorer. However, I think this might be a red herring. Generally, people move to a city to get higher income. If the market is roughly efficient, then the income boost from being in a city should at least offset the increase in cost of living. So it factors out of the equation.

View more: Next