Michael_S comments on Proposal for an Pre-registered Experiment in EA Outreach - Effective Altruism Forum

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (9)

You are viewing a single comment's thread.

Comment author: Michael_S 08 January 2017 04:54:57PM *  4 points [-]

This sounds really great to me. I love the idea of having more RCTs in the EA sphere. I would definitely record how much they are giving 1 year later.

I also think it's worth having a hold out set. People can pre-register the list of friends, than a random number generator can be used to randomly selects some friends not to make an explicit GWWC pitch to. It's possible many of the friends/contacts who join GWWC and start donating are those who have already been exposed to EA ideas before over a long period of time, and the effect size of the direct GWWC pitch isn't as large as it would appear. Having a hold out set would account for this. With a hold out set, CEA wouldn't have to worry about who they contact. The holdout set would take care of this and make the estimate of the treatment effect unbiased.

Comment author: Linch 09 January 2017 12:13:14AM 1 point [-]

I don't think there will be enough datapoints to do this. But if there are enough people willing to be in this experiment and they think they have a lot of friends they'll be willing to contact, I will include a holdout set.