W

weeatquince

5946 karmaJoined Sep 2014

Comments
463

I went though the old emails today and I am confident that my description accurately captured what happened and that everything I said can be backed up.

Another animal advocacy research organization supposedly found CE plagiarizing their work extensively including in published reports, and CE failed to address this.

Hi, I am Charity Entrepreneurship (CE, now AIM) Director of Research. I wanted to quickly respond to this point.

I believe this refers to an incident that happened in 2021. CE had an ongoing relationship with an animal advocacy policy organisation occasionally providing research to support their policy work. We received a request for some input and over the next 24 hours we helped that policy organisation draft a note on the topic at hand. In doing so a CE staff member copy and pasted text from a private document shared by another animal advocacy research organisation. This was plagiarism and should not have happened. I would like to note: firstly that this did not happen in the course of our business as usual research process but in a rushed piece of work that bypassed our normal review process, and secondly that this report was not directly published by us and it was not made clear to the CE staff member involved that the content was going to be made into a public report (most other work for that policy organisation was just used privately) although we should of course have considered this possibility. These facts of course do not excuse our mistake here but are relevant for assessing the risk that this was any more than a one-off mistake.

I was involved in responding when this issue came to light. On the day the mistake was realised we: acknowledged the mistake, apologised to the injured party, pulled all publicity for the report, drafted an email to the policy org asking to have the person who's text was copied added as a co-author (the email was not sent until the following day after as we waited for approval from the injured party). The published report was updated. Over the next three weeks we carried out a thorough internal risk assessment including reviewing all past reports by the same author. The other animal rights research organisation acknowledged they were satisfied with the steps taken. We found no cases of plagiarism in any other reports (the other research org agreed with this assessment), although one other tweak was made to a report to make acknowledgment more clear. 

FWIW I find mildlyanonymous' description of this event to be somewhat misleading referring to multiple "reports" and claiming "CE failed to address this".

 

CE's reports on animal welfare consistently contain serious factual errors ... noticing immediately that it had multiple major errors, sharing that feedback, and having it ignored due to their internal time capping practices.

I don’t know what this is about. I know of no case where we have ignored feedback. We are always open to receiving feedback on any of our reports from anyone at any time. I am very sorry if I or any CE staff ignored you and I open to hearing more about this, and/or hearing about any errors you have spotted in our research. If you can share any more information on this I can look into it, please contact me (I will PM you my email address, note I am about to go on a week's leave). It is often the case if we receive minor critical feedback after a report is published we do not go back and edit our report but note the feedback in our Implementation Note for future Charity Entrepreneurship founders, maybe that is what happened.

This is a great post and captured something that I feel. Thank you for writing it Michelle!!

Thank you for a nuanced and interesting reply.

Thank you so much for an excellent post.

I just wanted to pick up on one of your suggested lessons learned that, at least in my mind, doesn’t follow directly from the evidence you have provided.

You say:

These wins suggest a few lessons. ... the value of cross-party support. Every farm animal welfare law I’m aware of, globally, passed with cross-party support. ... We should be able to too: there are many more conservative animal lovers than liberal factory farmers.

To me, there are two very opposing ways you could take this. Animal-ag industry is benefiting from cross party support so:
A] Animal rights activists need to work more with the political right so that we get cross-party support too, essentially depoliticising animal rights policy, with the aim of animal activists also getting the benefits of cross party support.
B] Animal rights activists need to work more with the political left so that supporting animal farming is an unpalatable opinion or action for anyone on the left to hold, essentially politicising animal rights policy, with the aim of industry loosing the benefits of cross-party support.

Why do you suggest strategy A] depolticisation? Working with conservative animal lovers. Do you have any evidence this is the correct lesson to draw?

I have not yet done much analysis of this question but my initial sense from the history of social change in the US is that the path to major change though an issue becoming highly politicised and championed by one half of the political spectrum is likely to be the quicker (albeit less stable) route to success, and in some cases where entrenched interests are very strong, might be the only path to success (e.g. with slavery). I worry the a focus on depoliticsiation could be a strategic blunder. I have been pondering this for a while and am keen to understand what research, evidence and reasoning there is for keeping animal rights depoliticsed.

Have you considered blinded case work / decision making? Like one person collects the key information annonomises it and then someone else decides the appropriate responce without knowing the names / orgs of the people involved.

Could be good for avoiding some CoIs. Has worked for me in the past for similar situations.

Thank you Saulius. Very helpful to hear. This sounds like a really positive story of good management of a difficult situation. Well done to Marcus.

If I read between the lines a bit I get the impression that maybe more junior (be that less competent or just newer to the org) managers at Rethink with less confidence in their actions not rocking the Rethink<->funder relationship were perhaps more likely to put unwelcome pressure on researchers about what to publish. Just a hypothesis, so might be wrong. But also the kind of thing good internal policies, good onboarding, good senior example setting, or just discussions of this topic, can all help with. 

[This comment is no longer endorsed by its author]Reply

I found this reply made me less confident in Rethink's ability to address publication bias. Some things that triggered my 'hmmm not so sure about this' sense were:

  • The reply did not directly address the claims in Saulius's comment. E.g. "I'm sorry you feel that way" not "I'm sorry”. No acknowledgement that if, as Saulius claimed, a senior staff told him that it was wrong to have expressed his views to OpenPhil (when asked by OpenPhil for his views), that this might have been a mistake by that staff. I guess I was expecting more of a 'huh that sounds bad, let us look into it' style response than a 'nothing to see here, all is great in Rethink land' style response.
  • The story about Saulius' WAW post. Judging from Marcus' comment It sounds like Rethink stopped Saulius from posting a WAW post (within a work context) and it also looks like there was a potential conflict of interest here for senior staff as posting could affect funding. Marcus says that RP senior staff were only deciding based on the quality of the post. Now I notice the post itself (published a few months after Saulius left Rethink) seems to have significantly more upvotes than almost any of Rethink's posts (and lots of positive feedback). I recognise upvotes are not a great measure of research quality but it does make me worry that about the possibility that this post was actually of sufficient quality and the conflict of interest may have biased senior Rethink staff to stop the post being published as a Rethink post.
  • The stated donor engagement principles seem problematic. Eg: "there are also times when we think it is important for RP to speak with a unified voice to our most important donors" is exactly the kind of reason I would use if I was censoring staff's interaction with donors. It is not that "unified organisational voice" policies are wrong, just that without additional safeguards they have a risk of being abused, of facilitating the kinds of conflict of interest driven actions under discussion here. Also as Saulius mentions such policies can also be one-sided where staff are welcome to say anything that aligns with a certain worldview but need to get sign-off to disagree with that worldview, another source of bias.

I really really do like Rethink. And I do not think that this is a huge problem, or enough of a problem to stop donors giving to Rethink in most cases. But I would still be interested in seeing additional evidence of Rethink addressing this risk of bias. 

Hi Peter, Rethink Priorities is towards the top of the places I'm considering giving this year. This post was super helpful. And these projects look incredible, and highly valuable.

That said I have a bunch of questions and uncertainties I would love to get answers to you before donating to Rethink.


1. What is your cost/benefit. Specially I would love to know any or all of:

  • Rethink's cost per researcher year on average, (i.e. the total org cost divided by total researcher years, not salary).
  • Rethink's cost per published research report (again total org cost not amount spent on a specific projects, divided by the number of published reports where a research heavy EA Forum post of typical Rethink quality would count as a published report).
  • Rethink's cost per published research report that is not majority funded by "institutional support"

 

2. Can you assure me that Rethink's researchers are independent?

Rethink seems to be very heavily reliant for its existence on OpenPhil (and maybe some other very large institutional supporters?). I get the impression from a few people that Rethink's researchers sometimes may be restricted in their freedom to publish or freedom to say what they want to funders, where doing so would go against the wishes of Rethinks institutional funders (OpenPhil) or would in some way pose a risk to Rethink's funding stream.

This creates a risk of bias. E.g. a publication bias that skews Rethink to only publish information that OpenPhil are happy with. This is a key reason towards me feeling happier donating to support research from more independent sources (such as EA Infrastructure Fund or HLI or Animal Ask) rather than giving to Rethink.

I would be reassured if I saw evidence of Rethink addressing this risk of bias. This could look like any or all of:

  • A public policy that researchers have the freedom to publish even where donors disagree, and sets out steps to minimize the risk of bias (e.g. ignoring the views of donors when deciding if something is an info-hazard) and the accountability mechanisms to ensure that the policy is followed (e.g. whistleblowing process).
  • Some case studies of how Rethink has made this publication bias / upset funders trade-off in the past, especially evidence of Rethink going against OpenPhil's direct wishes.
  • An organisational risk register entry from Rethink setting out plans to mitigate these risks or why senior staff are not worried about these risk.
  • A policy from OpenPhil setting out an aim not to bias Rethink in this way.

 

3. How will you prioritise amongst the projects listed here with unrestricted funds from small donors?

Most of these projects I find very exciting, but some more than others. Do you have a rough priority ordering or a sense of what you would do in different scenarios, like if you ended up with unrestricted funding of $0.25m/$0.5m/$1m/$2m/$4m etc how you would split it between the projects you list?

 

4. [Already answered] Can you clarify how much your examples you cite are funded by OpenPhil?

[This was one of my key questions but Marcus answered it on another thread, I included it as I thought it would be relevant to others.]

I noted that you say that some research "invertebrate sentience, moral weights and welfare ranges, the cross-cause model, the CURVE sequence" has "historically not had institutional support". But OpenPhil suggest here they funded some of this work.

Marcus clarifies here that the Moral Weights work was about ~50% OpenPhil and the CURVE work about ~10% OpenPhil funded.  

 

Thank you for answering these questions. Keep up the wonderful work and keep improving our ability to do good in the world!!! <3 

 

[Minor edits made to q2]

Load more