Hide table of contents

18

If you have a broadly effective altruist approach, we think 80,000 Hours is an attractive donation opportunity from a number of perspectives.

In this post, we’ll sketch some of these perspectives, and our current funding needs. This post is intended as a summary – full details on our plans and progress are provided in our annual review, and more details on our historical impact are provided here.

Although our evidence of impact isn’t as robust as some other organisations, we think the balance of evidence makes a persuasive case that 80,000 Hours is among the best funding opportunities in the community.

Note that this post is intended for people who are already familiar with our work. If you’d like an introduction, see our annual review, or ask me to send a summary (direct.ben at 80000hours dot org).

The marginal multiplier perspective

We track our impact in terms of impact-adjusted significant plan changes (IASPC). In the last three months, we’ve tracked an average of 130 per month.

Donation multiplier

  • We think the marginal cost per plan change is under £250. If you include opportunity costs of staff time, that might increase to around £500.
  • Each IASPC is worth at least £7,500 in donations to GiveWell-recommended charities (in expectation, on average), making our multiplier 15-fold. This is because over 2016, 13% of IASPC took the Giving What We Can (GWWC) pledge due to engaging with the guide, and we expect this to continue. GWWC estimates that each pledge is worth £58,000 in NPV, adjusted for the counterfactual, time-discounting and drop-out, and 13% of that is £7500.
  • This is likely an underestimate of the donation multiplier because it doesn’t fully account for the chance of finding another new very large donor, even though we’ve advised multiple high-net-worth individuals in the past (defined as having over $100m), and influenced six large earning to give donors (defined as planning to donate over $300,000 per year by 2019)
  • We roughly estimate that the donors intend to give 29% of the donations will go to GiveWell-recommended charities, 70% to meta-charity and existential risk, and 1% to animal welfare.
  • See more detail of these estimates and caveats here.

For this reason, we think it’s very likely more effective to donate to 80,000 Hours than a GiveWell-recommended charity, insofar as we can maintain these levels of return, unless you require very strong evidence of effectiveness.

Compared to other meta-charities, we can also compete on our donation multiplier, though GWWC is higher. In addition, 80,000 Hours is probably a multiplier on other meta-charities and existential risk organisations.

Talent multiplier

However, the aim of 80,000 Hours is to solve talent gaps rather than funding gaps – under 30% of the plan changers seek to earn to give or take the pledge. Since we already justify our costs on the basis of additional donations, benefits besides donations come for “free”. But if you believe the effective altruism community is more bottlenecked by talent than funding, our impact on talent gaps is likely to be much greater.

£500 is very little money to significantly influence a young graduate’s career path, especially when you consider that many of the plan changes were made by top students at global top 20 universities. As one indication, average US college graduate earnings are about $68,000 (£54,000), so the market value of just one year of their labor to a top problem area is over 100-times as large as our costs.

Here are are some examples of how we’ve helped to solve pressing talent gaps over recent years, where we think it’s likely we can do something similar in 2017.

  • The following effective altruist organisations either wouldn't exist or would have been significantly delayed without 80,000 Hours: Global Priorities Project, Raising for Effective Giving, dotimpact, and Animal Charity Evaluators.
  • In 2016, 9 people took jobs at effective altruist organisations (Centre for Effective Altruism, Founder’s Pledge, Animal Charity Evaluators) who were significant plan changes, and say they’re significantly less likely to have the job without 80,000 Hours.
  • 2015-2016, we found 46 people who reported a plan change who (i) want to work on AI safety research (ii) are concerned by existential risks (iii) have studied a relevant quantitative subject, and (iv) switched towards this path due to us. We’ve prepared materials such as the AI safety syllabus and introduced them to mentors to help them enter. One has landed a job at a top AI organisation.
  • Based on a sample, 15% of the plan changers say they switched which global problem they want to work on, and now intend to work on our top problem areas (promoting effective altruism, global priorities research, AI safety research or biosecurity) and no other areas. This would be over 200 people in 2016. If you only care about the plan changers who made a switch into a top problem area, the cost is £3,300 per IASPC (£500/15%).

See more detail on these examples and our historical impact.

Growth perspective

We think the marginal multiplier perspective is not however, the main reason to donate to 80,000 Hours. Rather, the majority of the value of donations comes from them giving us the chance to become much bigger in the future. This is what the Open Philanthropy Project calls “hits based giving”, and what we called the “growth approach” to evaluating non-profits.

80,000 Hours could be far bigger than it is today. If we could reach most of the students at top 20 universities who care about social impact, then we expect to cause at least 3-times as many plan changes each year. And that would mean that in twenty years, 80,000 Hours would have influenced a significant fraction of leaders in science, politics, business and other fields.

From there, we could expand into a wider range of universities and ages (e.g. into people mid-career). We could also provide advice to people who don’t already focus on social impact, with the aim of tilting them in that direction (we already do this as a side effect).

For this reason, 80,000 Hours has more room to grow than many other meta-charities. For instance, Raising for Effective Giving has already reached a large fraction of poker players, so it’s going to be hard to get 10-times larger within that audience. (CEA and perhaps some seed funding opportunities seem like the main exceptions.) This is a major additional benefit over the multiplier.

In addition, as we grow, we expand our alumni community that donates back to us. Donations now let us accelerate the growth of this community, unlocking more donations in the future.

80,000 Hours may be the fastest growing meta-charity. In the last two years, the rate of IASPC per month has increased 20-fold, reaching 150 per month (on costs that only increased 40% p.a.). We were also admitted to Y Combinator’s non-profit program that assesses non-profits using the growth approach. Any assessment of 80,000 Hours as a donation opportunity should take into account the likelihood that the organisation is 10-times larger in a couple of years.

As we’ve grown, the maximum upside of supporting 80,000 Hours has gone down (in that we have less room to grow than before), however, the likelihood of that growth being realised has gone up. Three years ago, reaching the scale we’ve reached today seemed like a long-shot; whereas now we’re confident we can get another 3-times larger.

Greater than 3-fold growth could be achieved just by focusing on our initial target audience of altruistic graduates at top universities. We could reach most of this audience through paid marketing for about £60,000 per year, so realising this upside seems likely.1

Building the effective altruism community

Many on the forum think that building the effective altruism community is a top priority. We think you can make a reasonable case that donating to 80,000 Hours is one of the most effective ways to grow the community.

Our career advice introduces people to the key ideas of effective altruism, and encourages them to get involved in the community. Over the past 12 months, 80000hours.org was the largest effective altruist website by traffic. Our newsletter has more subscribers than all the other organisations put together (88,000+). So we’re introducing more people to the ideas than anyone else.

We think career decisions are one of the best moments to introduce someone to effective altruism because (i) people are open to major shifts (ii) little good advice already exists (iii) we can help people to have a more fulfilling, successful career too. Contrast this with other organisations that focus on asking people to give away their money. In addition, we're the only effective altruist organisation focused on career decisions, whereas there are many organisations focused on donations and running student groups.

Over 2016, from a sample, 74% of those who made plan changes aren’t already involved with the community, and 43% said they intend to “get more involved in the effective altruism community”, which would be 608 people. There’s evidence people actually do get more involved – 80,000 Hours was the largest and highest-quality source of applicants to EAG this year, when measured in terms of acceptance rate. As shown above, we’ve also acted as a donation multiplier on meta-charities, increasing the funding for effective altruist organisations.

80,000 Hours also helps existing community members find higher impact careers, and resolve talent gaps in the community. Based on a sample, 9% of the plan changes, say they’re already heavily involved in the community. This would be 126 effective altruist careers switched per year.

Finally, 80,000 Hours contributes research and key considerations to the community, which makes it more effective over the long-term. One major example is simply the idea that career choices are important – much of the community was previously highly focused on donations. Some other examples include: the concept of talent gaps; rules of thumb for coordination; replaceability and its importance; earning to give, its pros and cons, and who should do it; the quantitative problem framework; and the growth approach.

At the same time, we’ve done this with a comparatively small budget. In 2016, 80,000 Hours only had 22% of the budget of CEA and 6% of the budget of GiveWell.2 When compared in terms of “effective altruism community growth per dollar”, 80,000 Hours has a good claim to be one of the most effective organisations.

Reducing existential risk

We also see 80,000 Hours as helping to reduce existential risk. Right now, we think artificial intelligence is the most pressing form of existential risk (though we also encourage people to work on biosecurity and nuclear security). In Chapter 15 of Superintelligence, Bostrom proposes three key ways to mitigate existential risks from artificial intelligence (AI): (i) community capacity building (ii) strategy research (i.e. global priorities research) and (iii) technical safety research. Addressing these three priorities in reverse order:

  1. To contribute to technical safety research, we created a pipeline of about 50 potential technical AI researchers who are concerned by existential risks (as mentioned above).
  2. We encourage people to work on strategy research (which we call global priorities research) to work out how best to mitigate existential risk. The Global Priorities Project at Oxford likely wouldn’t exist without 80,000 Hours.
  3. We build the existential risk community. In part, we do this by building the effective altruism community (as covered above). But we also grow the AI community directly. For instance, the Assistant Director of the Future of Humanity Institute, Niel Bowerman, who does the day-to-day management of the organisation, was influenced by 80,000 Hours. There are also plan changers at many of the top AI companies and research institutes.
  4. As covered, we’re also a multiplier on donations to meta-charities and existential risk organisations.

If you would like to learn more about how 80,000 Hours helps to mitigate existential risks, email me (direct.ben at 80000hours dot org).

Room for more funding

In previous years, 80,000 Hours hasn’t had a large room for more funding. However, given our recent success, we now think we should aim to scale up more aggressively. Several recent hires have also given us more capacity to invest in marketing and hiring.

For this reason, we’d like to grow our budget faster than the rate at which our existing donors can increase their funding, so we’ll need to welcome new donors or we won’t make our targets.

As further evidence, a survey of meta-charity donors carried out by Open Phil and 80,000 Hours found that they expect to give about £4.5m this year, and not all will go to meta-charities. Given that CEA is aiming to raise £2.5m-£5m alone, the capacity of meta-charity donors is going to be used up this year. This means we need new meta-charity donors, or good meta opportunities will go unfunded.

This all means that donations to 80,000 Hours this year will have an especially important impact on how fast we can grow.

However, we also think many people put too much weight on room for more funding as a factor. Our marginal donors are effective altruists, so if you donate to us, at worst you free up another effective altruist to donate elsewhere. Unless you have better information or unusual values compared to the rest of the community, this isn’t a bad outcome. In fact, by donating quickly you’ll save us and other donors time, which lets the community be more effective as a whole. We wrote more about donation coordination here.

Funding targets

We’d like to raise at least £1.7m. With this funding, over 2017, the aim is to triple the rate of IASPC. Here are some of the priorities we’d pursue to do that:

  1. Dramatically improve the career reviews and problem profiles, so we have in-depth profiles of all the best options, including advice on how to enter each area. For instance, we’d like to have in-depth profiles covering the main policy options, working at effective non-profits, and tech entrepreneurship.
  2. Create mentor networks. The easiest, scalable way to get more high-value plan changes seems like it’s to create ways to spot high-potential people and get them involved with our community. This year, we used a form on our website to find people interested in AI safety research careers, found a group of 50 with the relevant background, and found someone to mentor them. We’d like to do something similar for other key areas mentioned above. We could also do more to connect people with the effective altruism community.
  3. Make the online guide more engaging. We think there’s a lot that could be done here. For instance, we could create high-quality video content instead of articles. If this went well, we could put the guide on a MOOC platform like Coursera, which we expect to double the rate of sign-ups, and may improve the completion rate.
  4. Scale up outreach with the aim of saturating most of our target universities. We’d start by doing intensive outreach to one university in spring, then apply what we learn to 5-10 universities in our annual September outreach campaign. There’s scope to double or triple the number of students who use our advice, which could hugely grow the rate of plan changes. We could use digital advertising to gain new subscribers from target universities for under £4 (more detail).

This is roughly how the target breaks down:

  • Cover our existing commitments to 6 full-time staff and freelancers over 2017 (£585k).
  • Maintain at least 12 months’ reserves to give us financial security (to have this Dec ‘17, we need funding to cover 2018, which is £670k).
  • Increase salaries about 30% to match comparable organisations and attract better new staff (£350k over two years).
  • Hire two additional entry-level staff members to work on writing career reviews, giving workshops, or design (or one senior staff member) (£175k over two years)
  • Expand our marketing budget (£165k).
  • Subtract from the target the £250k of cash we already have on hand.

(See more detail on our plans and our budgets in our annual review)

Even if we made this target, it wouldn’t exhaust our room for more funding. If we raised more, we could increase our reserves, which would make it easier to attract staff, or we could pursue these expansion opportunities more aggressively (e.g. hire more, larger marketing budget).

Why not donate to 80,000 Hours?

We see the following as some of the main arguments against donating to 80,000 Hours, and we suggest nearby alternatives.

  1. Our impact is spread out over several problem areas. If you’re confident that one particular area is much more effective than the others, then it may be better to support that area directly. For instance, if you mainly just care about animal welfare, consider supporting ACE, which is a top pick by Open Philanthropy’s program manager in that area. If you just care about global poverty, then GWWC might provide a higher multiplier, at least in terms of donations (though GWWC is now merged with CEA). If you’re focused on AI risks then consider MIRI or FHI, which were picked by Daniel Daniel and Nick Beckstead at Open Phil, as well as Larks on the forum (though we think 80,000 Hours is still a contender).

  2. If you don’t think the problems that the effective altruism community focuses on are among the most pressing, then we’re not having a large impact.

  3. If you’re unwilling to consider weak evidence of impact, or unable to spend enough time to investigate us to gain confidence in our effectiveness over first order charities, then it may be better to support something first order with stronger evidence behind it, such as GiveWell’s recommendations within global health. Similarly, if there's another meta-charity you understand much better, it may make sense just to donate to it.

  4. If you think some aspect of our career advice is badly wrong, such that it’ll cause some users to have far less impact. (If so, please tell us which part!)

  5. If your top cause is building the effective altruism movement, it’s plausible that creating better movement strategy, marketing and “infrastructure” (e.g. good local groups; a central community website) is the top priority right now. CEA is more focused on these activities than 80,000 Hours, so we’re keen to see CEA funded at least to the point where it can cover its basic activities in these areas.

  6. You think most of our impact comes from the effective altruism community rather than our programs, and that we haven’t done much to help the community. This could be true if you think the majority of the value of effective altruism community comes from word-of-mouth growth, and isn’t helped by organisations like 80,000 Hours.

  7. You think the average value of the plan changes is declining over time, in a way that’s not captured by the impact-adjustment of the metrics. The best argument we’ve heard is that most of the value comes from the plan changes rated “10” recruited via one-on-one outreach i.e. maybe what we score “10” is actually worth “50”. The 10s are growing much more slowly than the 1s, so if you have this view, our marginal impact is worse. We, however, think the 1:10 ratio is reasonable, and that growth in the 10s will pick up with some delay – 10s take longer than 1s.

Conclusion

80,000 Hours is an attractive donation opportunity purely on the basis of its marginal multiplier, both in terms of donations and talent. But in addition, we have major potential to grow, and as a side effect, we’re one of the most-effective groups growing the effective altruism community. We’re also attractive if you’re mainly concerned by existential risks, and we have more room for funding than we’ve had in recent years.

One of these perspectives by itself might not be convincing. The nature of our work makes it hard to quickly convey a compelling and robust case for impact. However, we think if you consider the balance of evidence, there’s a good case to be made.

If you’d like to help us grow in 2017, leave your email in this form.

If you’d like to learn more about our progress over 2016, see our annual review.

 

 

 


1. There’s about 200,000 undergraduates at the world’s top 20 universities. 25% choose careers each year, so that’s 50,000 per year. About 30% care about their social impact, which is 15,000. We estimate we can get newsletter subscribers at target universities for under £4, so the cost to get 15,000 is £60,000 per year. The cost to merely reach them would be much less.

2. In 2016, our budget was about £250,000.

CEA’s budget was £1,332,298. Archived link, retrieved 18 Dec 2016.

GiveWell’s budget was approximately $5m. Archived link, retrieved 18 Dec 2016.

We spent $3.0m in the 12 months from December 1, 2014 to November 30, 2015. We currently project expenses of $4.9m for December 1, 2015 to November 30, 2016.

18

0
0

Reactions

0
0

More posts like this

Comments18
Sorted by Click to highlight new comments since: Today at 3:23 PM

I'm glad that you write this sort of thing. 80K is one of the few organizations that I see writing "why you should donate to us" articles. I believe more organizations should do this because they generally know more about their own accomplishments than anyone else. I wouldn't take an organization's arguments as seriously as a third party's because they're necessarily biased toward themselves, but they can still provide a useful service to potential donors by presenting the strongest arguments in favor of donating to them.

I have written before about why I'm not convinced that I should donate to 80K (see the comments on the linked comment thread). I have essentially the same concerns that I did then. Since you're giving more elaborate arguments than before, I can respond in more detail about why I'm still not convinced.

My fundamental concern with 80K is that the evidence it its favor is very weak. My favorite meta-charity is REG because it has a straightforward causal chain of impact, and it raises a lot of money for charities that I believe do much more good in expectation than GiveWell top charities. 80K can claim the latter to some extent but cannot claim the former.

Below I give a few of the concerns I have with 80K, and what could convince me to donate.

Highly indirect impact. A lot of 80K's claims to impact rely on long chains such that your actual effect is pretty indirect. For example, the claim that an IASPC is worth £7500 via getting people to sign the GWWC pledge relies on assuming:

  • These people would not have signed the pledge without 80K.
  • These people would not have done something similarly or more valuable otherwise.
  • The GWWC pledge is as valuable as GWWC claims it is.

I haven't seen compelling evidence that any of these is true, and they all have to be true for 80K to have the impact here that it claims to have.

Problems with counterfactuals.

When someone switches from (e.g.) earning to give to direct work, 80K adds this to its impact stats. When someone else switches from direct work to earning to give, 80K also adds this to its impact stats. The only way these can both be good is if 80K is moving people toward their comparative advantages, which is a much harder claim to justify. I would like to see more effort on 80K's part to figure out whether its plan changes are actually causing people to do more good.

Questionable marketing tactics.

This is somewhat less of a concern, but I might as well bring it up here. 80K uses very aggressive marketing tactics (invasive browser popups, repeated asks to sign up for things, frequent emails) that I find abrasive. 80K justifies these by claiming that it increases sign-ups, and I'm sure it does, but these metrics don't account for the cost of turning people off.

By comparison, GiveWell does essentially no marketing but has still attracted more attention than any other EA organization, and it has among the best reputations of any EA org. It attracts donors by producing great content rather than by cajoling people to subscribe to its newsletter. For most orgs I don't believe this would work because most orgs just aren't capable of producing valuable content, but like GiveWell, 80K produces plenty of good content.

Perhaps 80K's current marketing tactics are a good idea on balance, but we have no way of knowing. 80K's metrics can only observe the value its marketing produces and not the value it destroys. It may be possible to get better evidence on this; I haven't really thought about it.

Past vs. future impact.

80K has made a bunch of claims about its historical impact. I'm skeptical that the impact has been as big as 80K claims, but I'm also skeptical that the impact will continue to be as big. For example, 80K claims substantial credit for about a half dozen new organizations. Do we have any reason to believe that 80K will cause more organizations to be created, and that they will be as effective as the ones it contributed to in the past? 80K's writeup claims that it will but doesn't give much justification. Similarly, 80K claims that a lot of benefit comes from its articles, but writing new articles has diminishing utility as you start to cover the most important ideas.


In summary, to persuade me to donate to 80K, you need to convince me that it has sufficiently high leverage that it does more good than the single best direct-work org, and it has higher leverage than any other meta org. More importantly, you need to find strong evidence that 80K actually has the impact it claims to have, or better demonstrate that the existing evidence is sufficient.

Hi Michael,

Thanks for the comments.

Our last chain got very long, and it didn't feel like we made much progress, so I'm going to limit myself to one reply.

My fundamental concern with 80K is that the evidence it its favor is very weak. My favorite meta-charity is REG because it has a straightforward causal chain of impact, and it raises a lot of money for charities that I believe do much more good in expectation than GiveWell top charities. 80K can claim the latter to some extent but cannot claim the former.

I agree that the counterfactuals + chain of impact are clearer with REG, however, strength of evidence is only one criteria I'd use when assessing a charity. To start with, I'm fairly risk-neutral, so I'm open to accepting weak evidence of high upside. But there's a lot of other considerations to use when comparing charities, including some of these:

  1. A larger multiplier - REG's multiplier is about 4x once you include opportunity costs (see figures https://80000hours.org/2016/12/has-80000-hours-justified-its-costs/#raising-for-effective-giving); whereas I've argued our multiplier at the margin is at least 15x.

  2. 80k's main purpose is to solve talent gaps rather than funding gaps, and about 70% of the plan changes don't donate. So probably most of the benefits of 80k aren't captured by the donation multiplier. We've also argued that talent gaps are more pressing than funding gaps. The nature of solving talent gaps means that your evidence will always be weak, so if donors only accept strong evidence, the EA community would never systematically deal with talent gaps.

  3. Better growth prospects - REG hasn't grown the last few years, whereas we've grown 20-fold; the remaining upside is also much better with 80k (due to the reasons in the post).

  4. Better movement building benefits. 80k is getting hundreds of people into the EA movement, who are taking key positions at EA orgs, have valuable expertise etc. REG has only got a handful of poker players into EA.

  5. 80k produces valuable research for the movement, REG doesn't.

  6. Donations to REG might funge with the rest of EAS, which mostly works on stuff with similar or even weaker evidence than 80k (EA movement building in German-speaking countries, Foundational Research Institute, animal-focused policy advocacy).

  7. Larger room for more funding. REG has been struggling to find someone to lead the project, so has limited room for funding. I'm keen to see that REG gets enough to cover their existing staff and pay for an Executive Director, but beyond that, 80k is better able to use additional funds.

On your specific criticisms of the donation multiplier calculation via getting people to pledge:

These people would not have signed the pledge without 80K.

We ask them whether they would have taken it otherwise, and don't count people who said yes. More discussion here: http://effective-altruism.com/ea/154/thoughts_on_the_meta_trap/9hk And here: https://80000hours.org/2016/12/has-80000-hours-justified-its-costs/#giving-what-we-can-pledges

These people would not have done something similarly or more valuable otherwise.

It seems like people take the GWWC pledge independently from their choice of job, so there isn't an opportunity cost problem here.

The GWWC pledge is as valuable as GWWC claims it is.

I'm persuaded by GWWC's estimates: https://www.givingwhatwecan.org/impact/ In fact, I think they undercount the value, because they ignore the chance of GWWC landing a super-large donor, which seems likely to be in the next 10 years. GWWC/CEA has already advised multiple billionaires.

On the opportunity cost point:

When someone switches from (e.g.) earning to give to direct work, 80K adds this to its impact stats. When someone else switches from direct work to earning to give, 80K also adds this to its impact stats.

Most of the plan changes are people moving from "regular careers" to "effective altruist style careers", rather than people who are already EAs switching between great options.

If someone switched from a good option to another good option, we may not count it as a plan change in the first place. Second, we wouldn't give it an equally high "impact rating".

In specific cases, we also say what the counterfactuals were e.g. see the end of the section on earning to give here: https://80000hours.org/2016/12/has-80000-hours-justified-its-costs/#the-earning-to-give-community

I would like to see more effort on 80K's part to figure out whether its plan changes are actually causing people to do more good.

In terms of whether people are going into the right options, our core purpose is research into which careers are highest impact, so that's where most of our effort already goes.

In terms of the specific cases, there's more detail here: https://80000hours.org/2016/12/has-80000-hours-justified-its-costs/

Questionable marketing tactics.

We're highly concerned by this, and think about it a lot. We haven't seen convincing evidence that we've been turning lots of people off (very unclear counterfactuals!). Our bounce rates etc. are if anything better than standard websites and I expect most people leave simply because they're not interested. Note as well that our intro advice isn't EA branded, which makes it much less damaging for the rest of the movement if we do make a bad impression.

We've also thought a lot about the analogy with GiveWell, and even discussed it with them. I think there's some important differences. Our growth has also been faster than GiveWell the last few years.

Do we have any reason to believe that 80K will cause more organizations to be created, and that they will be as effective as the ones it contributed to in the past?

4 new organisations seems like a pretty good track record - it's about one per year.

I expect more because we actively encourage people to set up new EA organisations (or work for the best existing ones) as part of our career advice.

Also note that our impact doesn't rest on this - we have lots of other pathways to impact.

but writing new articles has diminishing utility as you start to cover the most important ideas.

You can also get increasing returns. The more articles you have, the more impressive the site seems, so the more convincing it is. More articles can also bring the domain more traffic, which benefits all the other articles. More articles mean you cover more situations, which makes a larger fraction of users happy, increasing your viral coefficient.

You need to convince me that it has sufficiently high leverage that it does more good than the single best direct-work org, and it has higher leverage than any other meta org.

Yes. Though where it gets tricky is making the assessment at the margin.

Yes. Though where it gets tricky is making the assessment at the margin.

I was wondering about this too. Is your calculation of the marginal cost per plan change just the costs for 2016 divided by the plan changes in 2016? That doesn't seem to be an assessment at the margin.

Hi Rohin,

For the multipliers, I was trying to make a marginal estimate.

For the costs, there's a lot more detail here (it's not simply 2016 costs / 2016 #): https://80000hours.org/2016/12/has-80000-hours-justified-its-costs/#whats-the-marginal-cost-per-plan-change

However, like I say in the post, I'd caution against focusing too much on the marginal multiplier since I think more of our impact over 2017 will come from long-term growth rather the 2017 plan changes.

One margin is just running more workshops and that seems to have roughly the same marginal cost as our annual average. We just aren't close to running out of promising people interested in coming along.

To second this I was very surprised how little the attendance to the careers workshops we ran at Cambridge dropped off. I think we ended up doing five 200 person careers workshops before they stopped selling out.

If there is similar (or even only half as much) demand at other universities then there is a lot of opportunity to scale.

We actually were running out on other campuses until we figured out how to get online advertising to convert into workshop attendance - now feels we can do several times as many workshops as we're doing now without running out.

Do you have any info on how reliable self-reports are wrt counterfactuals about career changes and EWWC pledging?

I can imagine that people would not be very good at predicting that accurately.

One would expect some social acceptability bias that might require substantial creativity and work to measure.

Hi there,

It's definitely hard for people to estimate.

When we "impact rate" the plan changes, we also try to make an initial assessment of how much is counterfactually due to us (as well as how much extra impact results non-counterfactually adjusted).

We then to more in-depth analysis of the counterfactuals in crucial cases. Because we think the impact of plan changes it fat tailed, if we can understand the top 5% of them, we get a reasonable overall picture. We do this analysis in documents like this: https://80000hours.org/2016/12/has-80000-hours-justified-its-costs/

Each individual case is debateable, but I think there's a large enough volume of cases now to justify that we're having a substantial impact.

How did you come up with the estimate that 30% of undergraduates choosing careers care about their social impact?

I forgot to footnote it here, but there's more info in the main annual review doc under footnote 5: https://80000hours.org/2016/12/annual-review-dec-2016/#fn-5

Cheers for the update.

2015-2016, we found 46 people who reported a plan change who (i) want to work on AI safety research (ii) are concerned by existential risks (iii) have studied a relevant quantitative subject, and (iv) switched towards this path due to us.

How useful do these plan changes feel? e.g. How senior are these people? They could be ranked from 1 "Undergrad" - 2 "Applying for grad school" 3 "Grad school/starting to write papers", 4 "Postdoc" - 5 "Professor". There could be similar questions about how safety-oriented they are, how quantitatively oriented and how high-achieving they are generally.

Hey Ryan,

Good question. There's some stats here:

https://80000hours.org/2016/12/has-80000-hours-justified-its-costs/#technical-ai-safety-research-pipeline-of-50-people

Just let me know if you have more questions after that.

As further evidence, a survey of meta-charity donors carried out by Open Phil and 80,000 Hours found that they expect to give about £4.5m this year, and not all will go to meta-charities. Given that CEA is aiming to raise £2.5m-£5m alone, the capacity of meta-charity donors is going to be used up this year. This means we need new meta-charity donors, or good meta opportunities will go unfunded.

Is there more information about this survey currently available, and/or are there plans to release more information? This is the first I am hearing about the survey, and it sounds like something that deserves standalone coverage.

Hi Vipul,

I was planning to write up the results, but haven't been able to fit it in yet. Most of the information is confidential, so it needs some care.

I'm following up regarding this :).