AviN comments on Concerns with ACE research - Effective Altruism Forum

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (10)

You are viewing a single comment's thread.

Comment author: AviN 09 September 2018 11:18:34PM *  4 points [-]

I started out with a negative impression of ACE when I found it years ago. Since then, I've seen substantial improvements in their research quality, substantial willingness to update to new evidence, and substantial willingness to publicly state unpopular conclusions (e.g. leafletting has a slightly negative expected value). I was also impressed with the conference they ran in 2016. My overall impression is now positive, and I appreciate their contributions. I'd also suggest putting ACE in context: GiveWell, which generally has a positive reputation, also got off to a rough start.

I agree that ACE still has room to improve, and I appreciate that you have provided constructive feedback in good faith.

With regard to:

In its cost-effectiveness analyses, ACE estimates that their mean estimate of the “proportional improvement in welfare due to cage-free policies” is ~0.05, but provides only a one sentence explanation for this estimate.

I agree that ACE should provide more justification for this estimate, but I think there are a few points worth noting:

  • This is a fairly pessimistic estimate, far more pessimistic than a reasonable reading of De Mol et al 2006. (Of course a negative estimate would be even more pessimistic.)

  • GiveWell also makes some subjective judgements in their cost-effectiveness estimates that are not supported by comprehensive literature reviews, especially regarding moral weights.

With regard to:

In its review of THL’s and Animal Equality’s corporate outreach, ACE relies only on the charities’ self-reported corporate policy successes, which it then discounts by an arbitrary uncertainty factor: ~0.4 for both Animal Equality and THL.

I assume you're referring to the metric "THL's responsibility for changes"? My understanding that this is mostly supposed to reflect the case that it's often the case that multiple charities are involved in securing a given corporate commitment, making it incorrect to assign 100% of the effectiveness to just one charity.

With regard to:

ACE does not check with third party news sources, experts or with the companies themselves on whether the claims of the charities are accurate.

I agree that ACE should do this, but I predict most of the claims would withstand this scrutiny. As I mention here, I found that 15 out of the 22 corporate commitments that CIWF USA was allegedly involved in from January 2016 to March 2017 had some publicly available evidence to support their causal role.

With regard to grassroots outreach, it's worth noting that a large amount of THL's grassroots work in the past few years has been in coordinating in person protests against food corporations. This differs quite a bit from activities like leafletting, so it's understandable that evidence on the effectiveness of leafletting may not be the most relevant consideration to an evaluation of THL's grassroots work. (ACE has published an intervention report on protests earlier this year, which I haven't read yet.)