This is in response to Sarah Constantin's recent post about intellectual dishonesty within the EA community.
I roughly agree with Sarah's main object level points, but I think this essay doesn't sufficiently embody the spirit of cooperative discourse it's trying to promote. I have a lot of thoughts here, but they are building off a few existing essays. (There's been a recent revival over on Less Wrong attempting to make it a better locus for high quality discussion. I don't know if it's especially succeeded, but I think the concepts behind that intended revival and very important)
- Why Our Kind Can't Cooperate (Eliezer Yudkowsky)
- A Return to Discussion (Sarah Constantin)
- The Importance of [Less Wrong, OR another Single Conversational Locus] (Emphasis mine) (Anna Salamon)
- The Four Layers of Intellectual Conversation (Eliezer Yudkowsky)
I think it's important to have all three concepts in context before delving into: - EA has a lying problem (Sarah Constantin)
I recommend reading all of those. But here's a rough summary of what I consider the important bits. (If you want to actually argue with these bits, please read the actual essays before doing so, so you're engaging with the full substance of the idea)
- Intellectuals and contrarians love to argue and nitpick. This is valuable - it produces novel insights, and keeps us honest. BUT it makes it harder to actually work together to achieve things. We need to understand how working-together works on a deep enough level that we can do so without turning into another random institution that's lost it's purpose. (See Why Our Kind... for more)
- Lately, people have tended to talk on social media (Facebook, Tumblr, etc) rather than in formal blogs or forums that encourage longform discussion. This has a few effects. (See A Return to Discussion for more)
- FB discussion is fragmented - it's hard to find everything that's been said on a topic. (And tumblr is even worse)
- It's hard to know whether OTHER people have read a given thing on a topic.
- A related point (not necessarily in "A Return to Discussion" is that social media incentives some of the worst kinda of discussion. People share things quickly, without reflection. People read and respond to things in 5-10 minute bursts, without having time to fully digest them.
- Having a single, long form discussion area that you can expect everyone in an intellectual community to have read, makes it much easier to building knowledge. (And most of human progress is due, not to humans being smart, but being able to stand on the shoulders of giants). Anna Salamon's "Importance of a Single Conversational Locus" is framed around x-risk, but I think it applies to all aspects of EA: the problems the world faces are so huge that they need a higher caliber of thinking and knowledge-building than we currently have in order to solve.
- In order to make true intellectual progress, you need people to be able to make critiques. You also need those critics to expect their criticism to in turn be criticized, so that the criticism is high quality. If a critique turns out to be poorly thought out, we need shared, common knowledge of that so that people don't end up rehashing the same debates.
- And finally, (one of) Sarah's points in "EA has a lying problem" is that, in order to be different from other movements and succeed where they failed, EA needs to hold itself to a higher standard than usual. There's been much criticism of, say, Intentional Insights for doing sketchy, truth-bendy things to gain prestige and power. But that plenty of "high status" people within the EA community do things that are similar, even if to a different degree. We need to be aware of that.
I would not argue as strongly as Sarah does that we shouldn't do it at all, but it's worth periodically calling each other out on it.
Cooperative Epistemology
So my biggest point here, is that we need to be more proactive and mindful about how discussion and knowledge is built upon within the EA community.
To succeed at our goals:
- EA needs to hold itself to a very high intellectual standard (higher than we currently have, probably. In some sense anyway)
- Factions within EA needs to be able to cooperate, share knowledge. Both object level knowledge (i.e. how cost effective is AMF?) and meta/epistemic knowledge like:
- How do we evaluate messy studies
- How do we discuss things online so that people actually put effort into reading and contributing the discussion.
- What kinds of conversational/debate norms lead people to be more transparent.
- We need to be able to apply all the knowledge to go out and accomplish things, which will probably involve messy political stuff.
I have specific concerns about Sarah's post, which I'll post in a comment when I have a bit more time.
Note to casual viewers that the content of this is not what the title makes it sound like. He's not saying that rationalists are doomed to ultimately lie and cheat each other. Just that here are some reasons why it's been hard.
From the recent Sarah Constantin post
I don't buy this logic. Obviously there's a huge difference between taking power and then expending effort into positive activities, or taking power and not giving it up at all. Suppose that tomorrow we all found out that a major corporation was the front for a shady utilitarian network that had accumulated enough power and capital to fill all current EA funding gaps, or something like that. Since at some point you actually do accomplish good, it's clearly not indistinguishable.
I mean, you can keep kicking things back and say "why not secretly acquire MORE power today and wait till tomorrow, and then you'll never do any good?" but there's obvious empirical limitations to that, and besides it's a problem of decision theory which is present across all kinds of things and doesn't have much to do with gaining power in particular.
In practical terms, people (not EAs) who try to gain power with future promises of making things nicer are often either corrupt or corruptible, so we have that to worry about. But it's not sufficient to show that the basic strategy doesn't work.
...
{epistemic status: extremely low confidence}
The way I see a lot of these organizational problems where they seem to have controversial standards and practices is that core people are getting just a little bit too hung up on EA This and EA That and Community This and Community That... in reality what you should do is take pride in your organization, those few people and resources you have in your control or to your left and right, and make it as strong as possible. Not by cheating to get money or anything, but by fundamentally adhering to good principles of leadership, and really taking pride in it (without thinking about overall consequences all the time). If you do that, you probably won't have these kinds of problems, which seem to be kind of common whenever the organization itself is made subservient to some higher ideal (e.g. cryonics organizations, political activism, religions). I haven't been inside these EA organizations so I don't know how they work, but I know how good leadership works in other places and that's what seems to be different. It probably sounds obvious that everyone in an EA organization should run it as well as they can, but after I hear about these occasional issues I get the sense that it's kind of important to just sit and meditate on that basic point instead of always talking about the big blurry community.
I'd agree with all that. It all seems pretty reasonable.
I think that the main point here isn't that the strategy of building power and then do good never works, so much as that someone claiming that this is their plan isn't actually strong evidence that they're going to follow through, and that it encourages you to be slightly evil more than you have to be.
I've heard other people argue that that strategy literally doesn't work, making a claim roughly along the lines of "if you achieved power by maximizing influence in the conventional way, you wind up in an institutional context which makes pivoting to do ... (read more)