Buck comments on Why & How to Make Progress on Diversity & Inclusion in EA - Effective Altruism Forum

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (235)

You are viewing a single comment's thread.

Comment author: Buck 26 October 2017 04:24:49PM 22 points [-]

I think that your link to Georgia Ray's piece should make it clearer that her conclusion is

Studies don’t show that diversity has an overall clear effect, positive or negative, on the performance of teams or groups of people. (1) (2) The same may also be true on an organizational level. (3)

Your link implies that Georgia's post is overall positive on the effect of diversity on the performance of teams or groups, which I think is incorrect.

Comment author: gray 26 October 2017 07:41:00PM 16 points [-]

Georgia here - The direct context, "Research also shows that diverse teams are more creative, more innovative, better at problem-solving, and better at decision-making," is true based on what I found.

What I found also seemed pretty clear that diversity doesn't, overall, have a positive or negative effect on performance. Discussing that seems important if you're trying to argue that it'll yield better results, unless you have reason to think that EA is an exception.

(E.g., it seems possible that business teams aren't a good comparison for local groups or nonprofits, or that most teams in an EA context do more research/creative/problem-solving type work than business teams, so the implication "diversity is likely to help your EA team" would be possibly valid - but whatever premise that's based on would need to be justified.)

That said, obviously there are reasons to want diversity other than its effect on team performance, and I generally quite liked this article.

Comment author: Habryka 26 October 2017 10:18:01PM *  22 points [-]

As a relevant piece of data:

I looked into the 4 sources you cite in your article as improving the effectiveness of diverse teams and found the following:

  • 1 didn't replicate, and the replication found the opposite effect with a much larger sample size (which you link to in your article)
  • One is a Forbes article that cites a variety of articles, two of which I looked into and didn't say at all what the Forbes article said they say, with the articles usually saying "we found no significant effects"

  • One study you cited directly found the opposite result of what you seemed to imply it does, with its results table looking like this:


And the results section of the study explicitly saying:

"whereas background diversity displayed a small negative, yet nonsignificant, relationship with innovation (.133)."

(the thing that did have a positive relation was "job-related diversity" which is very much not the kind of diversity the top-level article is talking about)

  • The only study that you cited that did seem to cite some positive effects was one with the following results table:


Which found some effects on innovation, though overall it found very mixed effects of diversity, with its conclusion stating:

"Based on the results of a series of meta-analyses, we conclude that cultural diversity in teams can be both an asset and a liability. Whether the process losses associated with cultural diversity can be minimized and the process gains be realized will ultimately depend on the team’s ability to manage the process in an effective manner, as well as on the context within which the team operates."

Comment author: Halstead 27 October 2017 10:42:20AM 16 points [-]

I find this troubling. If a small sample of the evidence cited has been misreported or is weak, this seems to cast serious doubt on the evidence cited in the rest of the piece. Also, my prior is that pointing to lots of politically amenable social psychology research is a big red flag.

Comment author: casebash 26 October 2017 10:49:59PM 1 point [-]

So you research suggests that it improves creativity, innovation, problem solving and decision making, but not performance. That is a rather unexpected result. Do you have any thoughts on why this did not result in an improvement in total performance?

Comment author: Kelly_Witwicki 26 October 2017 04:48:46PM -3 points [-]

I didn’t mean to imply that — I just cited it as a source for the specific claims in that sentence. The other evidence I cite seems to imply it overall, and she doesn’t seem to account for all of that evidence.

I can’t tag here, but Georgia, if you see this I’d be curious for your opinion on how the totality of evidence weighs, particularly in expectation regardless of how robust it is.

Comment author: Buck 26 October 2017 05:13:04PM 23 points [-]

It feels like a bad practice to take a post which concludes that the effects are mixed or small, then just cite the effects in that post which seem positive and not mention the ones that seem negative or that the post overall disagrees with what you're trying to use it to argue for.

Comment author: Kelly_Witwicki 26 October 2017 07:55:07PM *  2 points [-]

That doesn't seem like what I'm doing. Georgia doesn't seem to be disagreeing with my post's overall argument (that EA would benefit from diversity; she actually seems to explicitly agree with that in her last paragraph), and she doesn't explicitly agree or disagree with the argument of that specific paragraph (that diversity tends to be net beneficial for groups). The quote you cite is about a "clear" effect on groups, from the evidence she evaluates, and I might not have the same bar for robustness that she's thinking of with that claim.

Moreover, her post argues

If we look at [the effect of diversity] further, we can decompose it into two effects (one where diversity has a neutral or negative impact on performance, and one where it has a mostly positive impact)

and goes onto explore these effects. The negative ones seem related to something like tribalism (e.g. less identification with the group), and I hope the EA community is able to overcome these avoidable downsides so it can on net benefit from diversity. I didn't mention them in the post because I think we can overcome them given our desire to de-bias ourselves, and given the tools that Georgia mentions we have to overcome them:

The more balanced a team is along some axis of diversity, the less likely you are to see negative effects on performance... recognition of less-obvious cognitive differences (e.g. personality and educational diversity) increases over time... diverse teams end up outperforming non-diverse teams [over time]... the longer a group works together, the less surface-level differences matter

I linked to her whole post so readers could see all of that. Linking directly to the citations I was pointing to in her post would have felt like cherry-picking. I could have given more explanation of her whole post in my own, and if I had spent more time writing this post, I probably would have done that.

[Edit: Georgia made a comment above that suggests she believes the statement without the robustness qualification, so we do have disagreement here.]