Kathy_Forth comments on An Exploration of Sexual Violence Reduction for Effective Altruism Potential - Effective Altruism Forum

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (133)

You are viewing a single comment's thread. Show more comments above.

Comment author: Marcus_N 14 November 2017 08:02:28AM *  1 point [-]

The case in this article draws heavily on the field of sexual violence research, but methodological problems in this field and premature thinking on the part of the author make this piece suffer from several problems: it skips over important methodological questions, misleads the audience about the rates of sexual violence, and advises hasty and socially punitive solutions.

It sounds like most of the audience hasn’t read the article closely and they are greatly underestimating the problems with it. As someone familiar with a lot of the literature referenced, my perception is that this article is recapitulating the standard feminist model of sexual assault, and attempting to insert this bottom line into an EA impact framework. My comment here is going to be a detailed multi-part rebuttal.

Sexual assault research is not valid

To understand what’s wrong with the validity) of the sexual assault research field, we will first examine the biasing incentives, and then we will break down the methodology of these studies in more detail.

Social science has a reputation for being a soft science, and sexual assault research is the softest corner of a softest field. Most of the researchers involved identify as political activists, feminists, or victim advocates. That’s not the most neutral approach. They get funding from government bodies with agendas, and their findings are used for political reasons: to expand the power of the federal government over universities, cause media outrage, create kangaroo courts, and to provide campaign-fodder for politicians. Sexual assault researchers have been caught committing outright fabrications and academic misconduct, which nonetheless become the basis for policy. Feminist researchers who produce high prevalence numbers will get lucrative consulting gigs or awards.

There is a strong incentive for sexual assault research to produce figures like “X% of women have been assaulted,” or “Y% of men admit to being rapists.” This article itself relies on these sorts of claims to present sexual assault as a common problem, and sexual assault prevention as a high-impact cause.

The original post cites figures like "36.3% of women and 17.1%” have been sexually assaulted. Elsewhere you, may see figures like “1 in 4” or “1 in 5” women have been raped. The article also cites a figure that 6% of men admit to committing rape.

Are these figures really credible, though? Of course not. Let’s take a look at these studies with our skeptic hats on.

You might be forgiven for thinking that 36.3% of women label themselves as sexually assaulted, or that 6% of men admit to committing rape. This is absolutely not how these surveys work. If people are asked such questions directly, there is a very low rate of agreement. For instance, in crime surveys, few women define themselves as a victim of the crime or rape or sexual assault.

Feminist researchers and activists had a bottom line: they believed that the true rate of rape was much higher, and that women were underreporting their victimization. So they started searching for a methodology to “prove” what they already believed, such as a Mary Koss’ “Sexual Experiences Survey”, which asks people if they were in certain situations, and then categorizes some of these situations as rape or sexual assault (based on conformance to a legal definition, or to the researchers’ own definition). This methodology generates much higher prevalence rates, which is why subsequent feminist researchers started using similar approaches. Very few people have pushed back against this approach, such as Christina Hoff Sommers. My argument here will be a more detailed version of hers.

So far we have reasons to distrust the field due to being greatly politicized or biased, but next we need to look at their actual methodology and see whether it is also biased. We can find the questions in the 2010 NIVS, Appendix C. I will excerpt some of them:

When you were drunk, high, drugged, or passed out and unable to consent, how many people ever…

  • had vaginal sex with you? By vaginal sex, we mean that {if female: a man or boy put his penis in your vagina}
  • {if male: a woman or girl made you put your penis in her vagina}? {if male}
  • made you perform anal sex, meaning that they made you put your penis into their anus?
  • made you receive anal sex, meaning they put their penis into your anus?
  • made you perform oral sex, meaning that they put their penis in your mouth or made you penetrate their vagina or anus with your mouth?
  • made you receive oral sex, meaning that they put their mouth on your {if male: penis} {if female: vagina} or anus?

Feminist sexual assault researchers count a “yes” on this question as evidence of sexual assault. But there are some obvious problems:

It’s ambiguous whether “unable to consent” modifies the whole list of situations, or just passed out. I think society generally agrees that sex with someone “passed out and unable to consent” is rape. But that question can be read in a way that someone could answer “yes” if they have had sex while drunk or high. But society does not agree with feminists that drunk or high sex is necessarily rape. In fact, it’s common for people to get drunk or high together as part of consensual sex.

Next question:

How many people have ever used physical force or threats to physically harm you to make you…

  • have vaginal sex?
  • {if male} perform anal sex?
  • receive anal sex?
  • make you perform oral sex?
  • make you receive oral sex?
  • put their fingers or an object in your {if female: vagina or} anus?

Surely an assent on this one is clearly sexual assault, right? Not so fast. Remember, we are reading this with our skeptic hats on, and our chivalrous hats off (and definitely with our feminist hats off). To people who are not bourgeois feminists, it’s likely that this is actually a loaded and convoluted question, which is not perfectly correlated with rape.

  • Physical force? It may shock feminist academics, but many consensual sexual relationships have some level of light physical forcefulness, which is sometimes referred to as “rough sex” or “passion.” Feminists themselves sometimes admit that sex is not always people gently caressing each other, on the condition that people engage in complex verbal rituals for consent if they want to get wilder. This is known as “BDSM” among people with high verbal IQ. However, much of the population doesn’t agree with feminists that rougher sex requires tedious verbal negotiations, and they negotiate it nonverbally. Sometimes this results in rape, but it is incorrect to equate physical force with rape by definition, because it’s part of the fabric of common sexual patterns. (As an extreme example, try to define consent in a mutually violent relationship, like portrayed here.

  • Similar problems with the wording of “make you.” Maybe it means rape, but it’s not analytically equivalent to “rape.” There is no guarantee that respondents understand it to mean rape. If the researchers were asking about rape, why not ask explicitly?

What is the false positive rate on these questions? 1%? 10%? 50%? Who knows, but it’s definitely not 0%. And that poses a problem for these prevalence figures, and any activists who are hawking them. They are being dishonest by not acknowledging this.

The same logic and methodological problems apply to the claim that 6% of men are admitted rapists. No, they aren’t. 6% of men answer convoluted questions on surveys that cause feminist researchers to categorize them as rapists. That’s not the same thing as being rapists.

Sexual assault prevalence rates are highly sensitive to methodology and operationalization. There may be no way to find the “true” rate of rape or sexual assault, because the answer to those questions is “it depends on how you ask the questions.” The way that feminists ask the questions is very convoluted and idiosyncratic, and an assent on a sexual violence survey has low external validity and does not generalize well to real-world situations.

If your female friend comes to you in real life and says that she was raped, then you have a large amount of information from her body language, and you can hear her story. This is a totally different level of evidence from someone checking a box on a survey and the researchers categorizing that response as rape.

Feminist researchers could have sidestepped a lot of these questions by using a less broad and more defensible definition of rape. For instance, they could ask “have you been raped,” or “have you ever raped someone.” There would be a lot less debate about what responses to those questions mean. However, this approach would have generated much lower prevalence numbers, and meant that the researchers got scooped by other feminist researchers using a broader definition. There is a general statistical problem that the larger the magnitude of your results, the less probable they are.

After looking at these studies with a critical eye, do EAs really want to bet that the approach here is optimal?

Comment author: Kathy_Forth 14 November 2017 08:32:51AM *  2 points [-]

I know about the replication crisis, I've read "Statistics Done Wrong" and I've read some Ioannidis. Perhaps I was too subtle, my way of addressing these concerns was to load up on as many review articles and meta-analyses as I could find, in all the areas where there was enough research for me to do so. In other areas, I looked for as many studies as I could find and included them all.

This is not perfect either. Ioannidis has warned about some specific vulnerabilities in meta-analyses and review articles. There isn't something perfect for me to do. I could have chosen to do nothing because the research is flawed. I decided that the subject is too important to ignore and I made the best of it.

A social sciences research disclaimer has been added. I thought that research quality issues were common knowledge in this social network. Maybe it is. Maybe that's mind projection fallacy. Now they have note about research quality.

Comment author: Marcus_N 15 November 2017 12:13:30AM *  3 points [-]

Truly acknowledging the problems with social science in general, and these studies in specific, would involve greatly softening your argument and shelving most of your prescriptions and impact analysis.

Sometimes, the best a field has to offer isn’t good enough to support policy recommendations, and epistemic humility requires acknowledging this. This body of research just isn’t strong enough to do the things that you want to do with it. I think that both you and the audience here has the sophistication to recognize the flaws in this research, and the lack of recognition is explained by biasing factors.

If you find feminist sexual assault research plausible based on your experiences, if you think that a large minority of men are rapists, paraphiliacs, harassers, or frotteurs, then that’s OK—everyone has their experience. But please mark your true reasons for believing those things, rather than acting like it’s scientifically solid enough to be a basis policy and community interventions. Then other people can make up their minds based on their own experiences.

Although you report following the review articles and meta-analyses, it is notable that all the evidence you discovered lines up perfectly with standard feminist narratives and your own experiences. From there, you jump into some very divisive prescriptions without seriously examining the countervailing evidence or the methodological problems. This is hasty at best, and morally questionable at worst—especially if, as you say, you were already aware of the methodological issues with social science.

Here are some of the things you either did not encounter or address. I am not claiming that you should have addressed any particular one of these, but they are critical parts of this debate:

  • The decades of criticism towards feminist rape statistics by Christina Hoff Sommers, among others.

  • The false accusation research.

  • The reality of how people communicate consent, which is primarily nonverbal and implicit, counter to the ideals of feminists and the BDSM community. This paper (unfortunately paywalled) cites a bunch of studies finding things like large percentages of women believing that they are giving consent to sex by doing nothing, or not resisting.

  • The problems of self-justification, unreliable memory, and sex differences in communication pointed out by feminist sociologist Carol Tavris.

  • The research on token resistance. Token resistance is often considered to be a misogynistic PUA red-pill rape myth, but it turns out that the concept has been extensively studied by feminist researchers, who find that women are often very internally divided about sex. One study found that 39% of women had engaged in token resistance, though later research complicated this figure. If indeed there is such a thing as token resistance, or “nonconsensual wanted sex”, or “consensual unwanted sex”, then this validates the common male complaints about women giving mixed signals, and Tavris’ claims about misunderstandings. It also causes significant problems for 3rd parties adjudicating he-said, she-said situations if communication is not straightforward.

  • The collapse of many of the high profile rape accusations backed by the media, such as the Rolling Stone story where a reporter went looking for a sympathetic rape victim, and the story fell apart. This proves that media coverage of sexual assault is inaccurate and biased towards feminism.

  • The college kangaroo courts for sexual assault in universities, which are based on the same sorts of scare-mongering statistics used in the original post. This leads to immense violations of due process (e.g. Sexual Assault Injustice at Occidental: College Railroads Accused Student, and Stanford Trains Student Jurors That ‘Acting Persuasive and Logical’ is Sign of Guilt). The policies are the direct result of the adoption of the same sorts of propagandistic statistics you are citing.

I would be surprised if you have never encountered any of these arguments, and you’ve never heard of Hoff Sommers, or the Rolling Stone story. If not, then your research is so deeply steeped in feminism that it has led you astray and caused you lose sight of any other perspectives. Addressing a single one of these issues would have caused your piece to be a lot more balanced. As it is, the original post comes off as the standard feminist anti-rape narrative inside a thin shell of effective altruism.