Comment author: Jc_Mourrat 07 September 2018 06:44:33PM *  12 points [-]

I had arrived at similar conclusions. Lately we were busy preparing material to put up on our freshly minted website at EA France. When it comes to recommendations, we naturally turn to charities such as GiveWell or ACE and refer to their work. But before putting these recommendations onto our website, I wanted to double-check that the evaluations are sound, because I want that we be ready to back them up publicly with confidence. So I tried to locate contradicting voices. For GiveWell, I found out that they disagree with other organizations such as the Campbell institute or Cochrane on the effectiveness of deworming programs. So I spent quite a bit of time reading the arguments of each party, and after that I came out extremely impressed by the depth and seriousness of the analysis of GiveWell. Of course I did not check all of what they do, but this experience gave me very high confidence that they are doing an outstanding work.

Then I moved to ACE. I took Nathan's article as a starting point for the disagreeing voice. Of course I was appalled by some of the points raised there, in particular in relation with leafletting. Also, a friend at AEF dig up a Facebook thread that happened around the time of publication of this article, and my recollection is that half of the people discussing this where just busy explaining that Nathan was really a very very mean person that we could not possibly imagine talking to.

I understand that this is old news, but I want to pause for a moment and reflect on the bigger picture. On human poverty, GiveWell is one among several very serious actors. It engages very thoroughly in discussions and explanations when diverging views emerge. We can argue about whether Nathan was diplomatic enough, etc, certainly he did not take all the precautions that Halstead has taken when writing this piece. But we have to realize that when it comes to animal suffering, as far as I know ACE is the only game in town. In my opinion, this is a precarious state of affairs, and we should do our best to protect criticism of ACE, even when it does not come with the highest level of politeness. Of course, I do not mean that people at ACE have bad intentions, but checks and balances are important, we are all human beings, and right now there seems to be precious little of these.

And as it turns out, as pointed out here by Halstead, at least some of the criticism of Nathan was actually correct, and is now acknowledged on the website of ACE (e.g. on leafletting).

And then I must stay that I fell off of my chair when I looked up the evaluation of corporate outreach and found out that the single argument in support of cage-free reform was this paper by De Mol et al (2016). To give an element of context, I have no previous exposure to animal wellfare, and yet it jumped at me that this was very bogus. How can this possibly happen?? I know that now ACE changed their position about this, but how they could come up with that in the first place, and how on Earth can this argument be still available online completely eludes me. All this while, as far as I can tell, corporate outreach is one of the flagship interventions advocated by ACE.

But again, I want to pause and think about the bigger picture for a while. The fact is that at the time of writing this argument, the organisation Direct Action Everywhere (DxE) had put up a rather comprehensive report explaining that they had come up with the opposite conclusion! (That cage-free reform is actually detrimental to animal wellfare.) I will refrain from discussing it at length here because this comment is already long, but this report of DxE was, in my opinion, dismissed with precious little good argument.

So again I see a small dissenting voice in the otherwise rather monopolistic position of ACE which is being dismissed without due consideration. And of course I get more worried. (To be perfectly clear, my point has nothing to do with whether DxE's conclusions were right or not; only with the fact that they were dismissed without proper consideration.)

I know that ACE no longer considers the article of De Mol et al (2016) as relevant, and things are clearly moving in a positive direction. Yet my confidence in ACE's work is at an all-time low. For full disclosure, I was planning to propose to my colleagues at AEF that we spend some time doing a "case study" in relation with the recommendations of ACE (similarly to studying the controversy about deworming for GiveWell). The "case study" I have in mind is the comparative evaluations of the Good Food Institute vs. New Harvest (As a side remark, I am interested in any previous discussion about this point.)

To sum up, I want to stress that I write this with the best intentions, and I appreciate that ACE has been improving a lot on all the points I have raised so far. Also, I understand that we cannot "go meta" and ask for evaluators of evaluators, evaluators of evaluators of evaluators, etc. Yet it is my impression that when it comes to human poverty, the situation is much, much healthier. In this area, there are several influential actors that have their own decision processes, can then compare their conclusions, and engage in serious debate about them. For animal suffering, I think it would do us a lot of good to make sure that dissenting voices are heard and protected, and that ACE engages with their arguments with much greater consideration than has been the case so far.

Comment author: AviN 10 September 2018 12:49:02AM 2 points [-]

On human poverty, GiveWell is one among several very serious actors. It engages very thoroughly in discussions and explanations when diverging views emerge.

The diverging views in the case of the GiveWell example you gave are from respected research organizations Campbell and Cochrane, with all parties arguing in good faith. This was very different from the case of Nathan's criticisms of ACE.

So again I see a small dissenting voice in the otherwise rather monopolistic position of ACE which is being dismissed without due consideration.

But ACE did reply to Nathan Harrison's criticisms:

https://animalcharityevaluators.org/blog/responses-to-common-critiques/ https://animalcharityevaluators.org/blog/response-to-a-recent-critique-of-our-research/

But again, I want to pause and think about the bigger picture for a while. The fact is that at the time of writing this argument, the organisation Direct Action Everywhere (DxE) had put up a rather comprehensive report explaining that they had come up with the opposite conclusion! (That cage-free reform is actually detrimental to animal wellfare.) I will refrain from discussing it at length here because this comment is already long, but this report of DxE was, in my opinion, dismissed with precious little good argument.

My understanding is that ACE did in fact take DxE's arguments into consideration, and that their relatively pessimistic estimate that cage-free represents a ~5% improvement is informed by many different views, including DxE's arguments that cage-free is harmful. (This is from conversations I've had and I'm not sure if ACE has published this. I agree it would be helpful if ACE published their reasons for this estimate.)

But we have to realize that when it comes to animal suffering, as far as I know ACE is the only game in town.

I'm not sure how you define "the only game in town." There are currently a number of other organizations who do research on effective animal advocacy, including Open Philanthropy, Sentience Institute, Rethink Charity, Faunalytics, Humane Society of the United States, Humane League Labs, Animal Welfare Action Lab, Wild Animal Suffering Research, etc.

Comment author: AviN 09 September 2018 11:18:34PM *  4 points [-]

I started out with a negative impression of ACE when I found it years ago. Since then, I've seen substantial improvements in their research quality, substantial willingness to update to new evidence, and substantial willingness to publicly state unpopular conclusions (e.g. leafletting has a slightly negative expected value). I was also impressed with the conference they ran in 2016. My overall impression is now positive, and I appreciate their contributions. I'd also suggest putting ACE in context: GiveWell, which generally has a positive reputation, also got off to a rough start.

I agree that ACE still has room to improve, and I appreciate that you have provided constructive feedback in good faith.

With regard to:

In its cost-effectiveness analyses, ACE estimates that their mean estimate of the “proportional improvement in welfare due to cage-free policies” is ~0.05, but provides only a one sentence explanation for this estimate.

I agree that ACE should provide more justification for this estimate, but I think there are a few points worth noting:

  • This is a fairly pessimistic estimate, far more pessimistic than a reasonable reading of De Mol et al 2006. (Of course a negative estimate would be even more pessimistic.)

  • GiveWell also makes some subjective judgements in their cost-effectiveness estimates that are not supported by comprehensive literature reviews, especially regarding moral weights.

With regard to:

In its review of THL’s and Animal Equality’s corporate outreach, ACE relies only on the charities’ self-reported corporate policy successes, which it then discounts by an arbitrary uncertainty factor: ~0.4 for both Animal Equality and THL.

I assume you're referring to the metric "THL's responsibility for changes"? My understanding that this is mostly supposed to reflect the case that it's often the case that multiple charities are involved in securing a given corporate commitment, making it incorrect to assign 100% of the effectiveness to just one charity.

With regard to:

ACE does not check with third party news sources, experts or with the companies themselves on whether the claims of the charities are accurate.

I agree that ACE should do this, but I predict most of the claims would withstand this scrutiny. As I mention here, I found that 15 out of the 22 corporate commitments that CIWF USA was allegedly involved in from January 2016 to March 2017 had some publicly available evidence to support their causal role.

With regard to grassroots outreach, it's worth noting that a large amount of THL's grassroots work in the past few years has been in coordinating in person protests against food corporations. This differs quite a bit from activities like leafletting, so it's understandable that evidence on the effectiveness of leafletting may not be the most relevant consideration to an evaluation of THL's grassroots work. (ACE has published an intervention report on protests earlier this year, which I haven't read yet.)

Comment author: AviN 09 September 2018 10:03:15PM *  4 points [-]

Thanks for the informative post, Toni!

With regard to:

We do search online for evidence in the news of each charity's achievements. The problem is: there usually is no such evidence, particularly in the field of corporate outreach. Of course, the absence of evidence of a charity's involvement in a corporate campaign is not evidence that the charity was not involved. We've also looked up corporations' press releases announcing their commitments, but these generally do not mention animal charities. (As far as I can remember, I've never seen one that does.) We have little reason to believe that a corporation could or would share detailed information about their decisions with us if we asked them. I don't know who Mr. Halstead has in mind when he mentions checking with "experts," though we've certainly spoken with many experts in corporate campaigning, if that is what he means.

I've found that the causal role of animal charities in corporate commitments is often supported by publicly available evidence. This evidence generally takes one of two forms:

  • Some corporations do name animal charities in their press releases. This often occurs when the charity secured the commitment through a cooperative approach, though it also sometimes occurs after a public campaign.

  • In cases of public campaigns, the timeline of events often provides some evidence of causality. I've found that the following pattern is typical: An animal charity launches a public campaign, leaving historical evidence in the form of a petition, tweets, media coverage, etc. Weeks or months later, the corporation publishes a press release agreeing to the commitment. (For reference, I've found Twitter to be a helpful resource for establishing these timelines.)

For example, here's is a list of corporate commitments that CIWF USA was allegedly involved in for the period from January 2016 to March 2017. (I had originally compiled this back in March 2017.) In 15 of the 22 cases, I found that their causal role was supported by publicly available evidence.

Comment author: saulius  (EA Profile) 08 August 2018 11:40:12PM 2 points [-]

I've just noticed that my text looks weird on mobile phone. I wrote it in google docs and pasted to EA forum. Is there any quick way to fix it? In case anyone has trouble reading it, you can also read it here.

Comment author: AviN 10 August 2018 04:29:08AM *  1 point [-]
Comment author: KevinWatkinson  (EA Profile) 23 July 2018 02:04:23PM 1 point [-]

I would agree, there's more scope beyond how the Open Philanthropy Welfare Fund presently operates so EA Funds has more potential utility there, but my own view is that the full range of possibilites aren't presently explored / considered because of time constraints alongside the low value of some disbursements alongside potentially having to spend more time justifying fairly unconventional grants.

In some ways i think it is the unconventional / marginal organisations which need more consideration as bringing potential value to the table over what is generally considered. Particularly in the way that a narrow funding focus could develop associations with particular organisations / ideas and so there could be issues of gravitating toward type.

I'm not sure what the solution is, perhaps another project worker at the Open Philanthropy Welfare Fund, maybe a small set of volunteers could be managed / empowered to work on building cases. It's difficult to know, but i do sympathise with the time constraints.

Comment author: AviN 26 July 2018 11:50:37PM 0 points [-]

Open Phil hired a Senior Associate, Farm Animal Welfare in March 2018.

https://www.openphilanthropy.org/about/team/amanda-hungerford

Comment author: kbog  (EA Profile) 25 April 2018 08:02:55PM *  7 points [-]

I didn't notice the community survey until I saw your comment. I had to retake the survey (answering "no my answers are not accurate") to get to it.

I think there will be selection bias when the survey is optional and difficult to access like this.

Comment author: AviN 26 April 2018 02:24:18PM 1 point [-]

I didn't see it either.

Comment author: jayquigley 20 April 2018 10:42:40PM *  3 points [-]

Another useful, well-writtten statement of this argument is in Brian Tomasik's "Does Vegetarianism Make a Difference?":

Suppose that a supermarket currently purchases three big cases per week of factory-farmed chickens, with each case containing 25 birds. The store does not purchase fractions of cases, so even if several surplus chickens remain each week, the supermarket will continue to buy three cases. This is what the anti-vegetarian means by "subsisting off of surplus animal products that would otherwise go to waste": the three cases are purchased anyway, so consuming one or two more chickens simply attenuates the surplus.

What would happen, though, if 25 customers decided to buy tempeh or beans instead of chickens? The purchasing agent who orders weekly cases of chickens would probably buy two cases instead of three. But any given consumer can't tell how far the store is from that cutoff point between three vs. two cases. The probability that any given chicken is the chicken that causes two cases instead of three to be purchased is 1/25. If you do avoid the chicken at the cutoff point, you prevent a whole case -- 25 chickens -- from being ordered next week. Thus, the expected value of any given chicken is (1/25) * 25 = 1 chicken, just like common sense would suggest.

Comment author: AviN 22 April 2018 01:39:38PM 1 point [-]

I wonder if the cutoff point is more like 25,000 though, the number of broiler chickens raised in a shed. It's unclear to me whether producers respond to small changes in demand by adjusting the numbers of broilers in a shed or only by adjusting the number of sheds in use.

If the cutoff point is more like 25,000, then this would imply that most veg*ns go their entire lives without preventing the existence of a single broiler through their consumption changes, while a minority prevent the existence of a huge number.

For what it's worth, it seems likely that donations to AMF are similar since their distributions typically cover hundreds of thousands or millions of people.

Comment author: AviN 05 April 2018 12:13:09AM *  4 points [-]

I agree with a lot of the content here, but I disagree with this suggestion:

The funds could take a similar approach to Giving What We Can – allocate funds to the top charities in their cause area, and donate to those charities on a regular basis until the fund manager comes along and updates the allocation.

If the annual discount rate is 12% and funds are granted out annually, then I believe this implies an average loss of around 6%. But the expected loss of a grant to a default charity vs a carefully selected charity is likely to be far greater than 6%.

Comment author: HaukeHillebrandt 03 April 2018 09:31:14AM 0 points [-]

Yes, absolutely, but the elasticity is somewhat hard to calculate (somewhat should try this though!). My example from above is just making a conservative assumption that the replacement effect is extreme. Of course it could be that there would have been a 37 million hen increase independent of corporate campaigns and that corporate campaigns have moved 25.8 million of those hens out of cages.

Comment author: AviN 03 April 2018 03:20:59PM *  1 point [-]

I mentioned this in a previous comment, but in case readers missed it:

  • The increase in flock size from December 2015 to December 2017 is far better explained by the US egg industry's recovery from an avian influenza outbreak than by cage-free pledges.

  • Norwood and Lusk (2011) estimate based on price elasticity data that, on the margin, a reduction in demand for 1 conventional egg causes a reduction in supply of 0.91 conventional eggs. But correspondingly, an increase in demand for 1 cage-free egg should lead to an increase in supply of less than 1 cage-free egg. So it's unclear why we should expect the transition to cage-free to increase the number of layer hens. If anything, the increase in prices caused by the transition should reduce the number of layer hens.

Comment author: HaukeHillebrandt 02 April 2018 11:23:19AM *  4 points [-]

This is very interesting thanks!

These projections of cost-effectiveness seem promising. I have a nagging related worry about what these campaigns have achieved so far, both in order to estimate a lower bound of their effectiveness, but which might also be relevant for future effectiveness. This worry resulted from the hypothesis that there is a displacement effect so that consumers and companies who buy cage free, will lower the price of caged eggs and thus increase demand from other consumers and retailers (in the US and potentially abroad).

Looking very briefly at the data it seems that the number of US cage free hens seem to have gone up in absolute terms by 25.8 million between Jan 2016-Oct 2017. However, it seems that total layer hens in the very similar time period from Dec '15 to Dec '17 have gone up by 37 million ( spreadsheet with sources ). In other words, the absolute number of caged hens seems to be increasing and corporate campaigns might have not had any effect at all so far. This seems to be in line with industry news.

This is also worrying especially if processed eggs from caged eggs might be exported to other countries in the future if the prices for eggs are further pushed down, or if processed eggs from caged hens are imported into the US.

But I'm not an export on this topic, so I would really like to hear someone to tell me what's wrong with this argument.

Comment author: AviN 03 April 2018 01:29:51PM *  2 points [-]

Hauke,

The layer hen flock size in December 2015 was unusually low because of an avian flu outbreak, so I don't think changes between December 2015 and December 2017 tell us much about the transition to cage-free.

It's also not clear why we should expect a transition to cage-free to increase the total number of layer hens based on price effects. If a buyer buys one less conventional egg, the expected supply of conventional eggs should fall by less than one because of price effects. On page 223 of Compassion by the Pound (2011), Norwood and Lusk estimate a decline of 0.91 eggs. But correspondingly, if a buyer buys one more cage-free egg, the increase in supply should increase by less than one, let's say 0.91 as well. I think it's a mistake to only consider price effects for the fall demand for conventional eggs for but not for the increase in demand for cage-free eggs. Of course it's fair to be on the lookout for evidence that one effect is stronger than the other, but the increase in egg supply from 2015 to 2017 is far better explained by a recovery from an avian influenza outbreak so I don't think it provides any meaningful evidence on this issue.

And if we consider price effects overall, the fact that cage-free egg production is somewhat more expensive than conventional egg production should cause a small decline in overall demand for eggs.

I think there is a risk that food corporations will renege on their pledges, perhaps arguing that the cage-free egg supply is insufficient. The parameter "Probability that groups will follow through on the pledges that they made" in ACE's estimates appear intended to capture this risk, and this is assigned a probability of 0.75 for THL in 2017. I think this risk underscores the importance of the work the animal organizations plan on follow-up with food corporations to ensure they follow through on their pledges, and that they start the transition early. I think this need for follow-up represents another limitation of ACE's cost-effectiveness estimates, since they assign all the benefits to the year of the pledges even though follow-up work will be required in future years.

Avi

View more: Next