(I work for EA Funds, including EAIF, helping out with public communications among other work. I'm not a grantmaker on EAIF and I'm not responsible for any decision on any specific EAIF grant). 

Hi. Thanks for writing this. I appreciate you putting the work in this, even though I strongly disagree with the framing of most of the doc that I feel informed enough to opine on, as well as most of the object-level. 

Ultimately, I think the parts of your report about EA Funds are mostly incorrect or substantively misleading, given the best information I have available. But I think it’s possible I’m misunderstanding your position or I don’t have enough context. So please read the following as my own best understanding of the situation, which can definitely be wrong. But first, onto the positives:

• I appreciate that the critical points in the doc are made as technical critiques, rather than paradigmatic ones. Technical critiques are ones that people are actually compelled to respond to, and can actually compel action (rather than just make people feel vaguely bad/smug and don’t compel any change).
• The report has many numerical/quantitative details. In theory, those are easier to falsify.
• The report appears extensive and must have taken a long time to write.

There are also some things the report mentioned that we have also been tracking, and I believe we have substantial room for improvement:

• Our grant evaluation process is still slower than we would like.
  • While broadly faster than other comparable funds I’m aware of (see this comment), I still think we have substantial room for improvement.
• Our various subfunds, particularly EAIF, have at points been understaffed and under-capacity. 
  • While I strongly disagree with the assessment that hiring more grantmakers is “fairly straightforward” (bad calls with grant evaluations are very costly, for reasons including but not limited to insufficient attention to adversarial selection; empirically most EA grantmaking organizations have found it very difficult to hire), I do think on the margin we can do significantly more on hiring.
• Our limited capacity has made it difficult for us to communicate with and/or coordinate with all the other stakeholders in the system, so we’re probably missing out on key high-EV opportunities (eg several of our existing collaborations in AI safety have started later than they counterfactually could have, and we haven’t been able to schedule time to fly to coordinate with folks in London/DC/Netherlands/Sweden.
  • One of the reasons I came on to EA Funds full-time is to help communicate with various groups. 

Now, onto the disagreements:

Procedurally:

• I was surprised that so much of the views that were ascribed to “EA Funds’ leadership” were from notes taken from a single informal call with the EA Funds project lead, that you did not confirm was okay to be shared publicly. 
  • They said that they usually explicitly request privacy before quoting them publicly, but are not sure if they did so in this instance.
  • They were also surprised that there was a public report out at all.
    • My best guess is that there was a misunderstanding that arose from a norm difference, where you come from the expectation that professional meetings are expected to be public unless explicitly stated otherwise, whereas the norm that I (and I think most EAs?) are more used to is that 1-1 meetings are private unless explicitly stated otherwise. 
  • They also disagree with the characterization of almost all of their comments (will say more below), which I think speaks to the epistemic advantages of confirming before publicly attributing comments made by someone else.
• I’d have found it helpful if you shared a copy of the post before making it public.
  • We could’ve corrected most misunderstandings
  • If you were too busy for private corrections, I could’ve at least written this response earlier.
• Many things in general were false (more details below). Having a fact-checking process might be useful going forwards.

Substantively:

• When the report quoted “CEA has had to step in and provide support in evaluating EAIF grants for them” I believe this is false or at least substantively misleading.
  • The closest thing I can think about is that we ask CEA Comm Health for help in reviewing comm health issues with our grants (which as I understand is part of their explicit job duties and both sides are happy with)”
    • It’s possible your source misunderstood the relationship and thought the Comm Health work was supererogatory or accidental? 
  • We frequently ask technical advisors for advice on project direction in a less institutionalized capacity as well^[1]. I think “step in” conveys the wrong understanding, as most of the grant evaluation is still done by the various funds.
  • (To confirm this impression, we checked with multiple senior people involved with CEA’s work; however, this was not an exhaustive sweep and it’s definitely possible that my impression is incorrect). 
• While it is true that we’re much slower than we would like, it seems very unreasonable to single out EA Funds grantmaking as "unreasonably long" when other grantmakers are as slow or slower. 
  • See e.g. Abraham Rowe’s comment here. 
    • “"Not weighing in on LTFF specifically, but from having done a lot of traditional nonprofit fundraising, I'd guess two months is a faster response time than 80% of foundations/institutional funders, and one month is probably faster than like 95%+. My best guess at the average for traditional nonprofit funders is more like 3-6 months. I guess my impression is that even in the worst cases, EA Funds has been operating pretty well above average compared to the traditional nonprofit funding world (though perhaps that isn't the right comparison). Given that LTFF is funding a lot of research, 2 months is almost certainly better than most academic grants.
    • My impression from what I think is a pretty large sample of EA funders and grants is also that EA Funds is the fastest turnaround time on average compared to the list you mention [Editor's note: "Open Phil, SFF, Founders Pledge, and Longview"] ([with] exceptions in some cases in both directions for EA Funds and other funders)”
    • I also sanity-checked with both Google Search and GPT-4^[2]
  • Broadly I’m aware that other people on the forum also believes that we’re slow, but I think most people who believe this believe so because:
    • We have our own aspirations to be faster, and we try to do so.
    • They think from a first-principles perspective that grantmakers “can” be faster.
    • We talk about our decisions and process very publicly, and so become more of an easy target for applicants’ grievances.
  • But while I understand and sympathize with other people’s frustrations^[3], it is probably not factually true that we’re slower in relative terms than other organizations, and it’s odd to single out EA Funds here.
• When your report said, “Critically, the expert reports that another major meta donor found EA Funds leadership frustrating to work with, and so ended up disengaging from further meta grantmaking coordination” my guess is that the quoted position is not true.
  • I feel like we’re keeping tabs on all the major donors (OP, SFF, Longview, etc).So I’m not sure who they could possibly be referring to.
    • Though I guess it’s possible that there is a major donor that’s so annoyed with us that they made efforts to hide themselves from us so we haven’t even heard about them.
    • But I think it’s more likely that the person in question isn’t a major donor.
  • To be clear, any donor feeling frustrated with us is regrettable. While it is true that not all donors can or should want to work with us (e.g. due to sufficiently differing cause prioritization or empirical worldviews), it is still regrettable that people have an emotionally frustrating experience. 
• Your report says that EA Funds leadership was “strongly dismissing the value of prioritization research, where other grantmakers generally expressed higher uncertainty”, but this is false.
  • I want to be clear that EA Funds has historically been, and currently is, quite positive on cause prioritization in general (though of course specific work may be lower quality, or good work that’s not cause prioritization may be falsely labeled as cause prioritization).
  • By revealed preferences, EAIF has given very large grants to worldview investigations and moral weights work at Rethink Priorities
  • By stated preferences, “research that aids prioritization across different cause areas” was listed as one of the central examples of things that EAIF would be excited to fund.
  • My understanding is that the best evidence you have for this view is that EA Funds leadership “would consider less than half of what [Rethink Priorities] does cause prioritization.” 
  • I’m confused why you think the quoted statement is surprising or good evidence, given that the stated claim is just obviously true. Eg, eg The Rodenticide Reduction Sequence or Cultured meat: A comparison of techno-economic analyses or Exposure to Lead Paint in Low- and Middle-Income Countries (To give three examples of work that I have more than a passing familiarity with) are much more about intervention prioritization than intercause prioritization. An example of the latter is the moral weights work at Rethink Priorities Worldview Investigations.
  • My best guess is that this is just a semantics misunderstanding, where EA Funds’ project lead was trying to convey a technical point about the difference between intercause cause prioritization vs intervention prioritization, whereas you understood his claim as a emotive position of “boo cause prioritization”
• Your report states that “EA Funds leadership doesn't believe that there is more uncertainty now with EA Fund's funding compared to other points in time” This is clearly false. 
  • I knew coming on to EA Funds that the job will have greater job uncertainty than other jobs I’ve had in the past, and I believe this was adequately communicated to me. 
  • We think about funding uncertainty a lot. EA Funds’ funding has always been uncertain, and things have gotten worse since Nov 2022.
  • Nor would it be consistent with either our stated or revealed preferences. 
    • Our revealed preference is that we spend substantially more staff time on fundraising than we have in the past.
    • Our stated preferences include “I generally expect our funding bar to vary more over time and to depend more on individual donations than it has historically.” and “LTFF and EAIF are unusually funding-constrained right now” 
      • The last one was even the title of a post with 4000+ views!
      • I don’t have a good idea for how much more unambiguous we could be.

Semantically:

I originally want to correct misunderstandings and misrepresentations of EA Funds’ positions more broadly in the report. However I think there were just a lot of misunderstandings overall, so I think it's simpler for people to just assume I contest almost every categorization of the form “EA funds believes X”. A few select examples:

• When your report claimed “​​leadership is of the view that the current funding landscape isn't more difficult for community builders” I a) don’t think we’ve said that, and b) to the extent we believe it, it’s relative to eg 2019; it’d be false compared to the 2022 era of unjustly excessive spending.
• “The EA Funds chair has clarified that EAIF would only really coordinate with OP, since they're reliably around; only if the [Meta-Charity Funders] was around for some time, would EA Funds find it worth factoring into their plans. ” 
  • To clarify, at the time the fund chair was unsure if MCF was only going to have one round. If they only have one round, it wouldn't make sense to change EA Funds’ strategy based on that. If they have multiple rounds (eg, more than 2, it could be worth factoring in). The costs of coordination are nontrivially significant.
  • It’s also worth noting that the fund chair had 2 calls with MCF and passed on various grants that they thought MCF might be interested in evaluating, which some people may consider coordination.
  • We’ve also coordinated more extensively with other non-OP funders, and have plans in the works for other collaborations with large funders.
• “In general, they don't think that other funders outside of OP need to do work on prioritization, and are in general sceptical of such work. ”
  • AFAIK nobody at EA Funds believes this.
• [EA funds believes] “so if EA groups struggle to raise money, it's simply because there are more compelling opportunities available instead.”
  • The statement seems kind of conceptually confused? Funders should always be trying to give to the most cost-effective projects on the margin.
  • The most charitable position that’s similar to the above I could think of is that some people might believe that 
    • “most community-building projects that are not funded now aren’t funded because of constraints on grantmaker capacity,” so grantmakers make poor decisions
      • Note that the corollary to the above is that many of the community building projects that are funded should not have been funded. 
    • I can’t speak to other people at EA Funds, but my own best guess is that this is not true (for projects that people online are likely to have heard of).
      • I’m more optimistic about boutique funding arrangements for projects within people’s networks that are unlikely to have applied to big funders, or people inspiring those around them to create new projects.
  • If projects aren’t funded, the biggest high-level reason is that there are limited resources in the world in general, and in EA specifically. You might additionally also believe that meta-EA in general is underfunded relative to object-level programs. 
• (Minor): To clarify “since OP's GHW EA team is focusing on effective giving, EAIF will consider this less neglected” we should caveat this by saying that less neglected doesn’t necessarily mean less cost-effective than other plausible things for EAIF to fund.
• Re: “EA Funds not posting reports or having public metrics of success”
  • We do post public payout reports.
  • My understanding is that you are (understandably) upset that we don’t have clear metrics and cost-effectiveness analyses written up. 
    • I think this is a reasonable and understandable complaint, and we have indeed gotten this feedback before from others and have substantial room to improve here.
  • However, I think many readers might interpret the statement as something stronger, e.g. interpreting it as us not posting reports or writing publicly much at all
    • As a matter of practice, we write a lot more about what we fund and our decision process than any other EA funder I’m aware of (and likely more than most other non-EA funders). I think many readers may get the wrong impression of our level of transparency from that comment.

Note to readers: I reached out to Joel to clarify some of these points before posting. I really appreciate his prompt responses! Due to time constraints, I decided to not send him a copy of this exact comment before posting publicly.

1. ^{^}

   I personally have benefited greatly from talking to specialist advisors in biosecurity.

2. ^{^}

   From GPT4

   “The median time to receive a response for an academic grant can vary significantly depending on the funding organization, the field of study, and the specific grant program. Generally, the process can take anywhere from a few months to over a year. ”
   “The timeline for receiving a response on grant applications can vary across different fields and types of grants, but generally, the processes are similar in length to those in the academic and scientific research sectors.”
   “Smaller grants in this field might be decided upon quicker, potentially within 3 to 6 months [emphasis mine], especially if they require less funding or involve fewer regulatory hurdles.”

3. ^{^}

   Being funded by grants kind of sucks as an experience compared to e.g. employment; I dislike adding to such frustrations. There are also several cases I’m aware of where counterfactually impactful projects were not taken due to funders being insufficiently able to fund things in time, in some of those incidences I'm more responsible than anybody else.

Meta note: I wouldn’t normally write a comment like this. I don’t seriously consider 99.99% of charities when making my donations; why single out one? I’m writing anyway because comments so far are not engaging with my perspective, and I hope more detail can help 80,000 hours themselves and others engage better if they wish to do so. As I note at the end, they may quite reasonably not wish to do so.

For background, I was one of the people interviewed for this report, and in 2014-2018 my wife and I were one of 80,000 hours’ largest donors. In recent years it has not made my shortlist of donation options. The report’s characterisation of them - spending a huge amount while not clearly being >0 on the margin - is fairly close to my own view, though clearly I was not the only person to express it. All views expressed below are my own.

I think it is very clear that 80,000 hours have had a tremendous influence on the EA community. I cannot recall anyone stating otherwise, so references to things like the EA survey are not very relevant. But influence is not impact. I commonly hear two views for why this influence may not translate into positive impact:

-80,000 hours prioritises AI well above other cause areas. As a result they commonly push people off paths which are high-impact per other worldviews. So if you disagree with them about AI, you’re going to read things like their case studies and be pretty nonplussed. You’re also likely to have friends who have left very promising career paths because they were told they would do even more good in AI safety. This is my own position.

-80,000 hours is likely more responsible than any other single org for the many EA-influenced people working on AI capabilities. Many of the people who consider AI top priority are negative on this and thus on the org as a whole. This is not my own position, but I mention it because I think it helps explain why (some) people who are very pro-AI may decline to fund.

I suspect this unusual convergence may be why they got singled out; pretty much every meta org has funders skeptical of them for cause prioritisation reasons, but here there are many skeptics in the crowd broadly aligned on prioritisation.

Looping back to my own position, I would offer two ‘fake’ illustrative anecdotes:

Alice read Doing Good Better and was convinced of the merits of donating a moderate fraction of her income to effective charities. Later, she came across 80,000 hours and was convinced by their argument that her career was far more important. However, she found herself unable to take any of the recommended positions. As a result she neither donates nor works in what they would consider a high-impact role; it’s as if neither interaction had ever occurred, except perhaps she feels a bit down about her apparent uselessness.

Bob was having impact in a cause many EAs consider a top priority. But he is epistemically modest, and inclined to defer to the apparent EA consensus- communicated via 80,000 hours - that AI was more important. He switched careers and did find a role with solid - but worse - personal fit. The role is well-paid and engaging day-to-day; Bob sees little reason to reconsider the trade-off, especially since ChatGPT seems to have vindicated 80,000 hours’ prior belief that AI was going to be a big deal. But if pressed he would readily acknowledge that it’s not clear how his work actually improves things. In line with his broad policy on epistemics, he points out the EA leadership is very positive on his approach; who is he to disagree?

Alice and Bob have always been possible problems from my perspective. But in recent years I’ve met far more of them than I did when I was funding 80,000 hours. My circles could certainly be skewed here, but when there’s a lack of good data my approach to such situations is to base my own decisions on my own observations. If my circles are skewed, other people who are seeing very little of Alice and Bob can always choose to fund.

On that last note, I want to reiterate that I cannot think of a single org, meta or otherwise, that does not have its detractors. I suspect there may be some latent belief that an org as central as 80,000 hours has solid support across most EA funders. To the best of my knowledge this is not and has never been the case, for them or for anyone else. I do not think they should aim for that outcome, and I would encourage readers to update ~0 on learning such.

Hey, Arden from 80,000 Hours here – 

I haven't read the full report, but given the time sensitivity with commenting on forum posts, I wanted to quickly provide some information relevant to some of the 80k mentions in the qualitative comments, which were flagged to me.

Regarding whether we have public measures of our impact & what they show

It is indeed hard to measure how much our programmes counterfactually help move talent to high impact causes in a way that increases global welfare, but we do try to do this.

From the 2022 report the relevant section is here. Copying it in as there are a bunch of links. 

We primarily use six sources of data to assess our impact:

1. Open Philanthropy EA/LT survey. 
2. EA Survey responses. 
3. The 80,000 Hours user survey. A summary of the 2022 user survey is linked in the appendix. 
4. Our in-depth case study analyses, which produce our top plan changes and DIPY estimates (last analysed in 2020). 
5. Our own data about how users interact with our services (e.g. our historical metrics linked in the appendix). 
6. Our and others' impressions of the quality of our visible output. 

Overall, we’d guess that 80,000 Hours continued to see diminishing returns to its impact per staff member per year. [But we continue to think it's still cost-effective, even as it grows.]

Some elaboration: 

• DIPY estimates are our measure of contractual career plan shifts we think will be positive for the world. Unfortunately it's hard to get an accurate read on counterfactuals and response rates, so these are only very rough estimates & we don't put that much weight on them.
• We report on things like engagement time & job board clicks as *lead metrics* because we think they tend to flow through to counterfactual high impact plan changes, & we're able to measure them much more readily.
• Headlines from some of the links above: 
  • From our own survey (2138 respondents):
    • On the overall social impact that 80,000 Hours had on their career or career plans, 
      • 1021 (50%) said 80,000 Hours increased their impact
        • Within this we identified 266 who reported >30% chance of 80,000 Hours causing them to taking a new jobs or graduate course (a “criteria based plan change”)
      • 26 (1%) said 80,000 Hours reduced their impact.
        • Themes in answers were demoralisation and causing career choices that were a poor fit
  • Open Philanthropy's EA/LT survey was aimed at asking their respondents "​​“What was important in your journey towards longtermist priority work?” – it has a lot of different results and feels hard to summarise, but it showed a big chunk of people considered 80k a factor in ending up working where they are.
  • The 2020 EA survey link says "More than half (50.7%) of respondents cited 80,000 Hours as important for them getting involved in EA". (2022 says something similar) 

Regarding the extent to which we are cause neutral & whether we've been misleading about this

We do strive to be cause neutral, in the sense that we try to prioritize working on the issues where we think we can have the highest marginal impact (rather than committing to a particular cause for other reasons). 

For the past several years we've thought that the most pressing problem is AI safety, so have put much of our effort there (Some 80k programmes focus on it more than others – I reckon for some it's a majority, but it hasn't been true that as an org we “almost exclusively focus on AI risk.” (a bit more on that here.))

In other words, we're cause neutral, but not cause *agnostic* - we have a view about what's most pressing. (Of course we could be wrong or thinking about this badly, but I take that to be a different concern.)

The most prominent place we describe our problem prioritization is our problem profiles page – which is one of our most popular pages. We describe our list of issues this way: "These areas are ranked roughly by our guess at the expected impact of an additional person working on them, assuming your ability to contribute to solving each is similar (though there’s a lot of variation in the impact of work within each issue as well). (Here's also a past comment from me on a related issue.) 

Regarding the concern about us harming talented EAs by causing them to choose bad early career jobs

To the extent that this has happened this is quite serious – helping talented people have higher impact careers is our entire point! I think we will always sometimes fail to give good advice (given the diversity & complexity of people's situations & the world), but we do try to aggressively minimise negative impacts, and if people think any particular part of our advice is unhelpful, we'd like them to contact us about it! (I'm arden@80000hours.org & I can pass them on to the relevant people.)

We do also try to find evidence of negative impact, e.g. using our user survey, and it seems dramatically less common than the positive impact (see the stats above), though there are of course selection effects with that kind of method so one can't take that at face value!

Regarding our advice on working at AI companies and whether this increases AI risk

This is a good worry and we talk a lot about this internally! We wrote about this here.

Very interesting report. It provided a lot of visibility into how these funders think.

Geographically, India may be a standout opportunity for getting talent to do research/direct work in a counterfactually cheap way.

I would have embraced that more in the past, but I'm a lot more skeptical of this these days and I think that if it were possible then it would have worked out. For many tasks, EA wants the best talent that is available. Top talent is best able to access overseas opportunities and so the price is largely independent of current location.

In terms of age prioritization, it is suboptimal that EA focuses more on
outreach to university students or young professionals as opposed to mid-career
people with greater expertise and experience.

I agree that there is a lot of alpha in reaching out to mid-career professionals - if you are able to successfully achieve this. This work is a lot more challenging - mid-career professionals are often harder to reach, less able to pivot and have less time available to upskill. Less people are able to do this kind of outreach because these professionals may take current students or recent grads less seriously. So for a lot of potential movement builders, focusing on students or young professionals is a solid play because it is the best combination of impact and personal fit.

The report writes "80,000 Hours in particular, has been singled out for its work not having a clear positive impact, despite the enormous sums they have spent.

As an on-the-ground community builder, I'm skeptical of your take. So many people that I've talked to became interested in EA or AI safety through 80,000 Hours. Regarding: "misleading the community that it is cause neutral while being almost exclusively focused on AI risk", I was more concerned about this in the past, but I feel that they're handling this pretty well these days. Would be quite interested to hear if you had specific ways you thought they should change. Regarding, causing early career EA's to choose suboptimal early career jobs that mess up their CVs, I'd love to hear more detail on this if you can. Has anyone written up a post on this?

On funding Rethink Priorities specifically – a view floated is that there is value in RP as a check on OP, and yet if OP doesn't think RP is worth funding beyond a certain point, it's hard to gainsay them

Rethink Priorities seems to be a fairly big org - especially taking into account that it operates on the meta-level - so I understand why OpenPhil might be reluctant to spend even more money there. I suspect that there's a feeling that even if they are doing good work, that meta work should only be a certain portion of their budge. I wouldn't take that as a strong signal.

In practice, it's unclear if the community building actually translates to money/talent moved, as opposed to getting people into EA socially.

As someone on the ground, I can say that community building has translated to talent moved at the very least. Money moved is less visible to me because a lot of people aren't shouting about the pledges from the rooftops, plus the fact that donations are very heavy-tailed. Would love to hear some thoughtful discussion of how this could be better measured.

What would interest OP might be a project about getting in more people who are doing the best things in GHW (e.g. Buddy Shah, Eirik Mofoss).

Very curious about what impressed Open Philanthropy about these people since I'm not familiar with their work. Would be keen to learn more though!

I’m confused by the use of the term “expert” throughout this report. What exactly is the expertise that these individual donors and staff members are meant to be contributing? Something more neutral like ‘stakeholder’ seems more accurate.

[with respect to talent] In terms of age prioritization, it is suboptimal that EA focuses more on outreach to university students or young professionals as opposed to mid-career people with greater expertise and experience.

I think this is conditional on the object-level positions available being reasonably well-suited to mid-career folks. For instance, job security becomes increasingly important as one ages. Even to the extent that successful mid-career folks face a risk of being let go in their current positions, they know they should be able to find another position in their field pretty easily. Likewise, positions in some cause areas may not pay well enough to attract much interest from successful mid-career professionals in certain geographic locations who have a kid and a mortgage. Those areas are probably better off with the current recruiting focus, at least in developed countries.