There’s a knock against prediction markets, here, too. A Metaculus forecast, in March of 2022 (the end of the period when one could make forecasts on this question), gave a 1.3% chance of FTX making any default on customer funds over the year. The probability that the Metaculus forecasters would have put on the claim that FTX would default on very large numbers of customer funds, as a result of misconduct, would presumably have been lower.

Metaculus isn't a prediction market; it's just an opinion poll of people who use the Metaculus website.

Since writing that post, though, I now lean more towards thinking that someone should “own” managing the movement, and that that should be the Centre for Effective Altruism.

I agree with this. Failing that, I feel strongly that CEA should change its name. There are costs to having a leader / manager / "coordinator-in-chief", and costs to not having such an entity; but the worst of both worlds is to have ambiguity about whether a person or org is filling that role. Then you end up with situations like "a bunch of EAs sit on their hands because they expect someone else to respond, but no one actually takes the wheel", or "an org gets the power of perceived leadership, but has limited accountability because it's left itself a lot of plausible deniability about exactly how much of a leader it is".

Update Apr. 15:  I talked to a CEA employee and got some more context on why CEA hasn't done an SBF investigation and postmortem. In addition to the 'this might be really difficult and it might not be very useful' concern, they mentioned that the Charity Commission investigation into EV UK is still ongoing a year and a half later. (Google suggests that statutory inquiries by the Charity Commission take an average of 1.2 years to complete, so the super long wait here is sadly normal.)

Although the Commission has said "there is no indication of wrongdoing by the trustees at this time", and the risk of anything crazy happening is lower now than it was a year and a half ago, I gather that it's still at least possible that the Commission could take some drastic action like "we think EV did bad stuff, so we're going to take over the legal entity that includes the UK components of CEA, 80K, GWWC, GovAI, etc.", which may make it harder for CEA to usefully hold the steering wheel on an SBF investigation at this stage.

Example scenario: CEA tries to write up some lessons learned from the SBF thing, with an EA audience in mind; EAs tend to have unusually high standards, and a CEA staffer writes a comment that assumes this context, without running the comment by lawyers because it seemed innocent enough; because of those high standards, the Charity Commission misreads the CEA employee as implying a way worse thing happened than is actually the case.

This particular scenario may not be a big risk, but the sum of the risk of all possible scenarios like that (including scenarios that might not currently be on their radar) seems non-negligible to the CEA person I spoke to, even though they don't think there's any info out there that should rationally cause the Charity Commission to do anything wild here. And trying to do serious public reflection or soul-searching while also carefully nitpicking every sentence for possible ways the Charity Commission could misinterpret something, doesn't seem like an optimal set-up for deep, authentic, and productive soul-searching.

The CEA employee said that they thought this is one reason (but not the only reason) EV is unlikely to run a postmortem of this kind.

My initial thoughts on all this: This is very useful info! I had no idea the Charity Commission investigation was still ongoing, and if there are significant worries about that, that does indeed help make CEA and EV’s actions over the last year feel a lot less weird-and-mysterious to me.

I’m not sure I agree with CEA or EV’s choices here, but I no longer feel like there’s a mystery to be explained here; this seems like a place where reasonable people can easily disagree about what the right strategy is. I don't expect the Charity Commission to in fact take over those organizations, since as far as I know there's no reason to do that, but I can see how this would make it harder for CEA to do a soul-searching postmortem.

I do suspect that EV and/or CEA may be underestimating the costs of silence here. I could imagine a frog-boiling problem arising here, where it made sense to delay a postmortem for a few months based on a relatively small risk of disaster (and a hope that the Charity Commission investigation in this case might turn out to be brief), but it may not make sense to continue to delay in this situation for years on end. Both options are risky; I suspect the risks of inaction and silence may be getting systematically under-weighted here. (But it’s hard to be confident when I don’t know the specifics of how these decisions are being made.)

I ran the above by Oliver Habryka, who said:

“I talked to a CEA employee and got some more context on why CEA hasn't done an SBF investigation and postmortem.”

Seems like it wouldn't be too hard for them to just advocate for someone else doing it?

Or to just have whoever is leading the investigation leave the organization.

In general it seems to me that an investigation is probably best done in a relatively independent vehicle anyways, for many reasons.

“My thoughts on all this: This is very useful info! I had no idea the Charity Commission investigation was still ongoing, and that does indeed help make CEA and EV’s actions over the last year feel a lot less weird-and-mysterious to me.”

Agree that this is an important component (and a major component for my models).

I have some information suggesting that maybe Oliver and/or the CEA employee's account is wrong, or missing part of the story?? But I'm confused about the details, so I'll look into things more and post an update here if I learn more.

I feel like "people who worked with Sam told people about specific instances of quite serious dishonesty they had personally observed" is being classed as "rumour" here, which whilst not strictly inaccurate, is misleading, because it is a very atypical case relative to the image the word "rumour" conjures.

I agree with this.

[...] I feel like we still want to know if any one in leadership argued "oh, yeah, Sam might well be dodgy, but the expected value of publicly backing him is high because of the upside". That's a signal someone is a bad leader in my view, which is useful knowledge going forward.

I don't really agree with this. Everyone has some probability of turning out to be dodgy; it matters exactly how strong the available evidence was. "This EA leader writes people off immediately when they have even a tiny probability of being untrustworthy" would be a negative update about the person's decision-making too!

"Just focus on the arguments" isn't a decision-making algorithm, but I think informal processes like "just talk about it and individually do what makes sense" perform better than rigid algorithms in cases like this.

If we want something more formal, I tend to prefer approaches like "delegate the question to someone trustworthy who can spend a bunch of time carefully weighing the arguments" or "subsidize a prediction market to resolve the question" over "just run an opinion poll and do whatever the majority of people-who-see-the-poll vote for, without checking how informed or wise the respondents are".

Knowing what people think is useful, especially if it's a non-anonymous poll aimed at sparking conversations, questions, etc. (One thing that might help here is to include a field for people to leave a brief explanation of their vote, if the polling software allows for it.)

Anonymous polls are a bit trickier, since random people on the Internet can easily brigade such a poll. And I wouldn't want to assume that something's a good idea just because most EAs agree with it; I'd rather focus on the arguments for and against.

4/4 Update: An EA who was involved in EA's early response to the FTX disaster has give me their take on why there hasn't yet been an investigation. They think EA leaders (at least, the ones they talked to a lot at the time) had "little to do with a desire to protect the reputation of EA or of individual EAs", and had more to do with things like "general time constraints and various exogenous logistical difficulties".

See this comment for a lot more details, and a short response from Habryka.

Also, some corrections: I said that "there was a narrow investigation into legal risk to Effective Ventures last year", which I think risks overstating how narrow the investigation probably was. I also said that Julia Wise had "been calling for the existence of such an investigation", when from her perspective she "listed it as a possible project rather than calling for it exactly". Again, see the comment for details.

Update Apr. 4: I’ve now spoken with another EA who was involved in EA’s response to the FTX implosion. To summarize what they said to me:

• They thought that the lack of an investigation was primarily due to general time constraints and various exogenous logistical difficulties. At the time, they thought that setting up a team who could overcome the various difficulties would be extremely hard for mundane reasons such as:
  • thorough, even-handed investigations into sensitive topics are very hard to do (especially if you start out low-context);
  • this is especially true when they are vaguely scoped and potentially involve a large number of people across a number of different organizations;
  • “professional investigators” (like law firms) aren’t very well-suited to do the kind of investigation that would actually be helpful;
  • legal counsels were generally strongly advising people against talking about FTX stuff in general;
  • various old confidentiality agreements would make it difficult to discuss what happened in some relevant instances (e.g. meetings that had Chatham House Rules);
  • it would be hard to guarantee confidentiality in the investigation when info might be subpoenaed or something like that;
  • and a general plethora of individually-surmountable but collectively-highly-challenging obstacles.
• They flagged that at the time, most people involved were already in an exceptionally busy and difficult time, and so had less bandwidth for additional projects than usual.
• A caveat here is that the EV board did block some people from speaking publicly during the initial investigation into EV's legal situation. That investigation ended back in the summer of 2023. 
• Julia Wise and Ozzie Gooen wrote on the EA Forum that this is a potentially useful project for someone to take on, which as far as this person knew isn’t something any EA leadership did or would try to stop, and the impression of the person I spoke to was that Julia and Ozzie indeed tried to investigate what reforms should happen, though the person I spoke to didn’t follow that situation closely.
• The person I spoke to didn't want to put words in the mouth of EA leaders, and their information is mostly from ~1 year ago and might be out of date. But to the extent some people aren't currently champing at the bit to make this happen, their impression (with respect to the EA leaders they have interacted with relatively extensively) is that this has little to do with a desire to protect the reputation of EA or of individual EAs.
• Rather, their impression is that for a lot of top EA leaders, this whole thing is a lot less interesting, because those EAs think they know what happened (and that it's not that interesting). So the choice is like "should I pour in a ton of energy to try to set up this investigation that will struggle to get off the ground to learn kinda boring stuff I already know?" And maybe they are underrating how interesting others would find it, but that made the whole idea not so important-seeming (at least in the early days after FTX’s collapse, relative to all the other urgent and confusing things swirling around in the wake of the collapse) from their perspective.

I vouch for this person as generally honest and well-intentioned. I update from the above that community leaders are probably less resistant to doing some kind of fact-finding inquiry than I thought. I’m hoping that this take is correct, since it suggests to me that it might not be too hard to get an SBF postmortem to happen now that the trial and the EV legal investigation are both over (and now that we’re all talking about the subject in the first place).

If the take above isn’t correct, then hopefully my sharing it will cause others to chime in with further objections, and I can zigzag my way to understanding what actually happened!

I shared the above summary with Oliver Habryka, and he said:

Hmm, I definitely don’t buy the "this has little to do with EA leadership desire to protect their reputation". A lot of the reason for the high standards is for PR reasons.

I think people are like "Oh man, doing a good job here seems really hard, since doing it badly seems like it would be really costly reputation-wise. But if someone did want to put in the heroic effort to do a good enough job to not have many downsides, then yeah, I would be in favor of that. But that seems so hard to do that I don’t really expect it to happen."

Like, the primary thing that seems to me the mistake is the standard to which any such investigation is being held to before people consider it net-positive.

I'll also share Ozzie Gooen's Twitter take from a few days ago:

My quick guess (likely to be wrong!)

• There are really only a few people "on top" of EA.
• This would have essentially been "these top people investigating each other, and documenting that for others in the community"
• These people don't care that much about being trusted in the community. They fund the community and have power
• the EA community really doesn't have much power over the funders/leaders.
• These people generally feel like they understand the problem well enough.

And, some corrections to my earlier posts about this:

• I said that "there was a narrow investigation into legal risk to Effective Ventures last year", which I think may have overstated the narrowness of the investigation a bit. My understanding is that the investigation's main goal was to reduce EV's legal exposure, but to that end the investigation covered a somewhat wider range of topics (possibly including things like COI policies), including things that might touch on broader EA mistakes and possible improvements. But it's hard to be sure about any of this because details of the investigation's scope and outcomes weren't shared, and it doesn't sound like they will be.
• I said that Julia Wise had "been calling for the existence of such an investigation"; Julia clarifies on social media, "I would say I listed it as a possible project rather than calling for it exactly."
  • Specifically, Julia Wise, Ozzie Gooen, and Sam Donald co-wrote a November 2023 blog post that listed "comprehensive investigation into FTX<>EA connections / problems" as one of four "projects and programs we'd like to see", saying "these projects are promising, but they're sizable or ongoing projects that we don't have the capacity to carry out". They also included this idea in a list of Further Possible Projects on EA Reform.

Fair! I definitely don't want to imply that there's been zero reflection or inquiry in the wake of FTX. I just think "what actually happened within EA networks, and could we have done better with different processes or norms?" is a really large and central piece of the puzzle.

The issue is that there are degrees of naiveness. Oliver's view, as I understand it, is that there are at least three positions:

• Maximally Naive: Buy nice event venues, because we need more places to host events.
• Moderately Naive: Don't buy nice event venues, because it's more valuable to convince people that we're frugal and humble than it is valuable to host events.
• Non-Naive: Buy nice event venues, because we need more places to host events, and the value of signaling frugality and humility is in any case lower than the value of signaling that we're willing to do weird and unpopular things when the first-order effects are clearly positive. Indeed, trying to look frugal here may even cause more harm than benefit, since:
  • (a) it nudges EA toward being a home for empty virtue-signalers instead of people trying to actually help others, and
  • (b) it nudges EA toward being a home for manipulative people who are obsessed with controlling others' perceptions of EA, as opposed to EA being a home for honest, open, and cooperative souls who prize doing good and causing others to have accurate models over having a good reputation.

Optimizing too hard for reputation can get you into hot water, because you've hit the sour spot of being too naive to recognize that many others can see what you're doing and discount your signals accordingly, but not naive enough to just blithely do the obvious right thing without overthinking it.

There are obviously cases where reputation matters for impact, but many people fall into the trap of fixating on reputation when they lack the skill to take into account enough higher-order effects.

(Of course, the above isn't the only reason people might disagree on the utility of event venues. If you think EA is mainly bottlenecked on research and ideas, then you'll want to gather people together to solve problems and share their thoughts. If you instead think EA's big bottleneck is that we aren't drawing in enough people to donate to GiveWell top charities, then you should think events are a lot less useful, unless maybe it's a very large event targeted at drawing in new people to donate.)