The usefulness of the "bad people" label is exactly my point here. The fact of the matter is some people are bad, no matter what excuses they come up with. For example, Adolf Hitler was clearly a bad person, regardless of his belief that he was the single greatest and most ethical human being who had ever lived. The argument that all people have an equally strong moral compass is not tenable.

More than that, when I say "Sam Altman is a bad person", I don't mean "Sam Altman's internal monologue is just him thinking over and over again 'I want to destroy the world'". It means "Sam Altman's internal monologue is really good at coming up with excuses for unethical behavior".

Like:

I'm not concerned that Dario Amodei will consciously think to himself: "I'll go ahead and press this astronomically net-negative button over here because it will make me more powerful". But he can easily end up pressing such a button anyway.

I would like to state, for the record, that if Sam Altman pushes a "50% chance of making humans extinct" button, this makes him a bad person, no matter what he's thinking to himself. Personally I would just not press that button.

If I had to guess, the EA community is probably a bit worse at this than most communities because A) bad social skills and B) high trust.

This seems like a good tradeoff in general. I don't think we should be putting more emphasis on smooth-talking CEOs—which is what got us into the OpenAI mess in the first place. 

But at some point, defending Sam Altman is just charlie_brown_football.jpg

In the conversations I had with them, they very clearly understood the charges against him and what he'd done. The issue was they were completely unable to pass judgment on him as a person.

This is a good trait 95% of the time. Most people are too quick to pass judgment. This is especially true because 95% of people pass judgment based on vibes like "Bob seems weird and creepy" instead of concrete actions like "Bob has been fired from 3 of his last 4 jobs for theft".

However, the fact of the matter is some people are bad. For example, Adolf Hitler was clearly a bad person. Bob probably isn't very honest. Sam Altman's behavior is mostly motivated by a desire for money and power. This is true even if Sam Altman has somehow tricked himself into thinking his actions are good. Regardless of his internal monologue he's still acting to maximize his money and power.

EAs often have trouble going "Yup, that's a bad person" when they see someone who's very blatantly a bad person.

"Trust but verify" is Reagan's famous line on this.

Most EAs would agree with "90% of people are basically trying to do the right thing". But most of them have a very difficult time acting as though there's a 10% chance anyone they're talking to is an asshole. You shouldn't be expecting people to be assholes, but you should be considering the 10% chance they are and updating that probability based on evidence. Maya Angelou wrote "If someone shows you who they are, believe them the first time".

As a Bayesian who recognizes the importance of not updating too quickly away from your prior, I'd like to amend this to "If someone shows you who they are, believe them the 2nd or 3rd time they release a model that substantially increases the probability we're all going to die".

The empirical track record is that the top 3 AI research labs (Anthropic, DeepMind, and OpenAI) were all started by people worried that AI would be unsafe, who then went on to design and implement a bunch of unsafe AIs.

"AI is powerful and uncontrollable and could kill all of humanity, like seriously" is not a complicated message.

Anything longer than 8 morphemes is probably not going to survive Twitter or CNN getting their hands on it. I like the original version ("Literally everyone will die") better.

In what sense? The problem of potential helium scarcity has been (effectively) solved in the last few years by just looking for more helium deposits.

In a sense, we are "running out" (because there's only a finite supply of Helium), we're just running out very, very slowly.

Actually, maybe I should clarify this. This is standard practice when you hire a decent statistician. We've known this since like... the 1940s, maybe?

But a lot of organizations and clinical trials don't do this because they don't consult with a statistician early enough. I've had people come to me and say "hey, here's a pile of data, can you calculate a p-value?" too many times to count. Yes, I calculated a p-value, it's like 0.06, and if you'd come to me at the start of the experiment we could've avoided the million-dollar boondoggle

 that you just created.

You’re completely correct! However, it’s worth noting this is standard practice (when the treatment makes up most of the cost, which it usually doesn’t). Most statisticians will be able to tell you about this.

So I think I have two comments:

1. It’s actually pretty neat you figured this out by yourself, and shows you have a decent intuition for the subject.
2. However, if you’re a researcher at any kind of research institution, and you run or design RCTs, this suggests an organizational problem. You’re reinventing the wheel, and need to consult with a statistician. It’s very, very difficult to do good research without a statistician, no matter how clever you are. (If you’d like, I’m happy to help if you send me a DM.)

Could be marketed as a medication for hypersomnia, narcolepsy, chronic fatigue, and ADHD.