This is in response to Sarah Constantin's recent post about intellectual dishonesty within the EA community.
I roughly agree with Sarah's main object level points, but I think this essay doesn't sufficiently embody the spirit of cooperative discourse it's trying to promote. I have a lot of thoughts here, but they are building off a few existing essays. (There's been a recent revival over on Less Wrong attempting to make it a better locus for high quality discussion. I don't know if it's especially succeeded, but I think the concepts behind that intended revival and very important)
- Why Our Kind Can't Cooperate (Eliezer Yudkowsky)
- A Return to Discussion (Sarah Constantin)
- The Importance of [Less Wrong, OR another Single Conversational Locus] (Emphasis mine) (Anna Salamon)
- The Four Layers of Intellectual Conversation (Eliezer Yudkowsky)
I think it's important to have all three concepts in context before delving into: - EA has a lying problem (Sarah Constantin)
I recommend reading all of those. But here's a rough summary of what I consider the important bits. (If you want to actually argue with these bits, please read the actual essays before doing so, so you're engaging with the full substance of the idea)
- Intellectuals and contrarians love to argue and nitpick. This is valuable - it produces novel insights, and keeps us honest. BUT it makes it harder to actually work together to achieve things. We need to understand how working-together works on a deep enough level that we can do so without turning into another random institution that's lost it's purpose. (See Why Our Kind... for more)
- Lately, people have tended to talk on social media (Facebook, Tumblr, etc) rather than in formal blogs or forums that encourage longform discussion. This has a few effects. (See A Return to Discussion for more)
- FB discussion is fragmented - it's hard to find everything that's been said on a topic. (And tumblr is even worse)
- It's hard to know whether OTHER people have read a given thing on a topic.
- A related point (not necessarily in "A Return to Discussion" is that social media incentives some of the worst kinda of discussion. People share things quickly, without reflection. People read and respond to things in 5-10 minute bursts, without having time to fully digest them.
- Having a single, long form discussion area that you can expect everyone in an intellectual community to have read, makes it much easier to building knowledge. (And most of human progress is due, not to humans being smart, but being able to stand on the shoulders of giants). Anna Salamon's "Importance of a Single Conversational Locus" is framed around x-risk, but I think it applies to all aspects of EA: the problems the world faces are so huge that they need a higher caliber of thinking and knowledge-building than we currently have in order to solve.
- In order to make true intellectual progress, you need people to be able to make critiques. You also need those critics to expect their criticism to in turn be criticized, so that the criticism is high quality. If a critique turns out to be poorly thought out, we need shared, common knowledge of that so that people don't end up rehashing the same debates.
- And finally, (one of) Sarah's points in "EA has a lying problem" is that, in order to be different from other movements and succeed where they failed, EA needs to hold itself to a higher standard than usual. There's been much criticism of, say, Intentional Insights for doing sketchy, truth-bendy things to gain prestige and power. But that plenty of "high status" people within the EA community do things that are similar, even if to a different degree. We need to be aware of that.
I would not argue as strongly as Sarah does that we shouldn't do it at all, but it's worth periodically calling each other out on it.
Cooperative Epistemology
So my biggest point here, is that we need to be more proactive and mindful about how discussion and knowledge is built upon within the EA community.
To succeed at our goals:
- EA needs to hold itself to a very high intellectual standard (higher than we currently have, probably. In some sense anyway)
- Factions within EA needs to be able to cooperate, share knowledge. Both object level knowledge (i.e. how cost effective is AMF?) and meta/epistemic knowledge like:
- How do we evaluate messy studies
- How do we discuss things online so that people actually put effort into reading and contributing the discussion.
- What kinds of conversational/debate norms lead people to be more transparent.
- We need to be able to apply all the knowledge to go out and accomplish things, which will probably involve messy political stuff.
I have specific concerns about Sarah's post, which I'll post in a comment when I have a bit more time.
I note Constantin's post, first, was extraordinary uncharitable and inflammatory (e.g. the title for the section discussing Wiblin's remark "Keeping promises as a symptom of Autism", among many others); second, these errors were part of a deliberate strategy to 'inflame people against EA'; third, this strategy is hypocritical given the authors (professed) objections to any hint of 'exploitative communication'. Any of these in isolation is regrettable. In concert they are contemptible.
{ETA: Although in a followup post Constantin states her previous comments which were suggestive of bad faith were an "emotional outburst", it did not reflect her actual intentions either at the time of writing or subsequently.}
My view is that, akin to Hostadter's law, virtues of integrity are undervalued even when people try to account for undervaluing them: for this reason I advocate all-but lexical priority to candour, integrity, etc. over immediate benefits. The degree of priority these things should be accorded seems a topic on which reasonable people can disagree: I recommend Elmore's remarks as a persuasive defence of according these virtues a lower weight.
'Lower', however, still means 'quite a lot': if I read Elmore correctly, her view is not one can sacrifice scrupulously honest communication for any non-trivial benefit, but that these norms should on occasion be relaxed if necessary to realise substantial gains. The great majority of EAs seem to view these things as extremely important, and the direction of travel appears to me that 'more respected' EAs tend to accord these even greater importance (see MacAskill; c.f. Tsipursky).
My impression is that EAs - both individually and corporately - do fairly well in practice as well as principle. As Naik notes, many orgs engage in acts of honesty and accountability supererogatory to secure funding. When they do err, they tend to be robustly challenged (often by other EAs), publicly admit their mistake, and change practice (all of the examples Constantin cites were challenged at the time, I also think of Harris's concerns with promotion of EA global, GPP's mistaken use of a Stern report statistic, and now ACE). Similar sentiments apply to an individual level: my 'anecdata' is almost the opposite of Fluttershy's extremely bad experience: I sincerely believe (and even more sincerely hope) that mine is closer to the norm than theirs.
In absolute terms, I don't think EA in toto has a 'lying problem' (or a 'being misleading', 'not being scrupulously honest' problem). It seems to do quite well at this, and the rate and severity of the mistakes I see don't cause great alarm (although it can and should do better). Although relative terms are less relevant, I think it does better than virtually any other group I can think of.
I offer some further remarks on issues raised by some of the examples given which do not fit into the 'lying problem' theme:
1) Ironic, perhaps, that the best evidence for Todd's remark on the 'costs of criticism' arise from the aftermath of a post which (in part) unjustly excoriates him for that particular remark. My impression is that bad criticism is on average much more costly than bad praise, and some asymmetry in how these are treated seem reasonable.
I do not know whether journalistic 'best practice' around 'right of reply' extends to providing the criticism in full to its subject - regardless, it seems good practice to adopt for the reasons Todd explains. I have done this with my co-contributors re. Intentional Insights, and I have run a (yet to be published) piece about MIRI by MIRI as it had some critical elements to it. Naturally, if a critic does not do this for whatever reason, it does not mean their criticism should be ignored (I have yet to see a case of criticism 'shunned' for these reasons) but I think this is a norm worth encouraging.
2) Nonetheless, it may not have been advisable for the head of one 'part' of CEA to bring this up in context of criticism addressed to another part of CEA. Issues around appropriate disclosure have been mentioned before. In addition, remarks by 'EA public figures' may be taken as indicative of the view of their organisations or EA in toto even if explicitly disclaimed as 'personal opinion only'. A regrettable corollary (as Gordon-Brown notes) is a chilling effect on 'EA public figures' refraining from making unguarded remarks publicly. The costs of not doing so may be worse: if EA grows further, we may collectively regret 'providing more ammunition' to external critics to misuse.
3) Given the social costs towards an individual critic, there may be benefit (by organisations or, better, an independent 'Grand Inquisition' collaboration) to canvass these anonymously. The commonest shared could then be explored further: this would be valuable whether they point to a common misconception or a common fault. In the meanwhile, anyone is welcome to disclose criticisms or concerns to me in confidence.
4) Certain practices could be more widely adopted by EA orgs - beyond recording predictions, a prominent 'mistakes' page (per Givewell) would be desirable, likewise scrupulous declaration of relevant conflicts of interests.
5) (I owe this to Carl Shulman). Donors could also pitch in by carefully evaluating empirical or normative claims made by particular EA organisations: Plant, Dickens, and Hoffman would all be laudable examples, and I hope both to contribute some of my own work to this genre and to encourage others to do likewise.