16

Is EA growing? A concrete study idea to find out (and a $100 offer for implementation)

With a lot of investment in intentionally growing the EA movement this year, I'm curious if some concrete activities have caused EA to grow in noticeable ways.

In particular, I'm interested in looking at the impact of...

  • The publication of “Doing Good Better” by Will Macaskill
  • The publication of “The Most Good You Can Do” by Peter Singer
  • EA Global
  • GiveWell announcing its new top charities + associated mass media
  • Giving What We Can’s pledge drive

...on

  • Google searches for “effective altruism”
  • Visits to the EA Forum
  • Amount of posts on the EA Forum
  • Number of people who have joined the EA Facebook group
  • Number of posts on the EA Facebook group
  • Visits to the GWWC website and sign-ups to their newsletter
  • Number of Giving What We Can pledges
  • Visits to the GiveWell site and sign-ups to their newsletter
  • Visits to 80,000 Hours website and sign-ups to their newsletter
  • Visits to effectivealtruism.org and sign-ups for the EA newsletter
  • Number of mentions in the press by people unaffiliated by EA
  • Number of people who have created an EA Profile or put themselves on the map of EAs
  • Visits to TLYCS website
  • Signups to TLYCS newsletter

-

I think doing this analysis could provide some good time-order evidence of causality between certain EA activities and the growth of the EA movement in different areas. It might also allow us to make further judgements about (a) whether certain effects are just “flashes in the pan” or sustainable and repeatable ways to grow the movement or (b) whether certain actions grow some parts of the EA movement but not others.

Of course, doing such an analysis is a bit reductive and the raw numbers might not be good enough to capture all the ways in which the EA movement grows, especially on key metrics we might care about, like growing existing EAs, growing donation totals, coming up with new EA ideas that improve the movement as a whole, and creating significant career changes. Still, I think having this analysis could be helpful for deciding where to invest in the EA movement.

Furthermore, it’s possible that some of the dependent variables (e.g., visits to the EA Forum) might be affected by things outside the event of interest (e.g., during the same time of EA Global but independently of EA Global, there is a campaign to post EA Forum articles on Reddit, which brings in traffic). This is a limitation of any time-order but non-experimental analysis, but the hope is by tracking enough variables that this limitation can be smoothed out.


I unfortunately don’t have much time to think about this or implement it myself, but I think it is important enough to put some money on it. I’m willing to offer...
  • $2 to anyone who suggests an event or variable to look at that I merit is worth considering
  • $10 to anyone who improves this study design itself in a way that causes me to update this document
  • $100 to whomever successfully implements this study and publishes the results publicly (though email me at peter@peterhurford.com if you intend to do so, to avoid duplication of effort).

All the data needed for the study should be readily available -- contact me by email if you need help accessing a particular data source and I can tell you where to look or whom to ask for the data.

The money will come in the form of either me giving the money to you over PayPal or through me donating to a charity of your choice. The money is coming from my personal savings and would counterfactually not have been used for any donations.

-

Update: Eric Yu did this, writing "Effects of major events on EA activity". (Here's my $100 donation.)

Comments (18)

Comment author: Ben_Todd 24 February 2016 08:59:06AM *  7 points [-]

The main things I've noticed at 80k:

  • We had some very cool people contact us after reading Will's book, who were potential staff members.
  • EAG has been useful for finding staff. Mostly because EAO had such a large team in the Bay, they ended up knowing and checking out lots of potential hires who I didn't already know. The list of applications to EAG was also a little useful.
  • I think we get slightly more traffic because the EA community promotes us more (e.g. more FB likes of posts because the EA FB group is larger; EA newsletter recommends our posts), but it's not a big factor in traffic growth.
  • The student groups helped us get lots of newsletter subs. We got 7-8k at the start of this academic year, vs. 3k last year. (though we also put a bunch more effort into it)

We also get a lot of benefit in being able to introduce new people to members of the community, and my sense is that the quality of the introductions we're able to make keeps improving.

Comment author: Gleb_T  (EA Profile) 22 February 2016 04:14:55AM 5 points [-]

I'd suggest further factors:

  • 1) Visits to TLYCS website; 2) Signups to their newsletter; 3) Growth of fans on their FB page; 4) Growth of followers on their Twitter profile; 5) Number of people using their Impact Calculator; 6) Number of people taking the TLYCS pledge
  • 1) Visits to GWWC website; 2) Signups to their newsletter; 3) Growth of fans on their FB page; 4) Growth of followers on their Twitter profile
  • 1) Visits to GiveWell website; 2) Signups to their newsletter; 3) Growth of fans on their FB page; 4) Growth of followers on their Twitter profile
  • 80K 1) Growth of fans on their FB page; 2) Growth of followers on their Twitter profile
  • Number of new EA local groups (LEAN can provide numbers, I'm sure)
  • Visits to effectivealtruism.org
  • Signups to EA newsletter
  • Signups to EA Local newsletter
  • Unsolicited email contacts sent to EA meta-charities
  • Donations to top GiveWell-ranked charities
  • Donations to EA meta-charities
Comment author: Peter_Hurford  (EA Profile) 23 February 2016 08:03:35PM *  1 point [-]

1) Visits to TLYCS website; 2) Signups to their newsletter; 3) Growth of fans on their FB page; 4) Growth of followers on their Twitter profile; 5) Number of people using their Impact Calculator; 6) Number of people taking the TLYCS pledge

Good idea. I'll add that.

-

1) Visits to GWWC website; 2) Signups to their newsletter; 3) Growth of fans on their FB page; 4) Growth of followers on their Twitter profile

I'll add that too. Is GWWC active on Twitter though? I think visits and newsletter sign-ups are likely good enough.

-

1) Visits to GiveWell website; 2) Signups to their newsletter; 3) Growth of fans on their FB page; 4) Growth of followers on their Twitter profile

Also a good idea. Do you know which FB page they have? Are they active on Twitter? Again, I think visits and newsletter sign-ups are likely good enough.

-

80K 1) Growth of fans on their FB page; 2) Growth of followers on their Twitter profile

-

Number of new EA local groups (LEAN can provide numbers, I'm sure)

My best guess is that these local groups would be driven mainly by LEAN's activities and less by the growth of the movement as a whole. I'd expect to be an important metric of movement growth but more of a lagging metric or an input rather than an output metric to track.

-

Visits to effectivealtruism.org; Signups to EA newsletter

Good idea.

-

Signups to EA Local newsletter

Which newsletter is that?

-

Unsolicited email contacts sent to EA meta-charities

This sounds really hard to track.

-

Donations to top GiveWell-ranked charities

Donations to EA meta-charities

My guess is that these would be lagging metrics that would be harder to track in time, but they'd definitely be good things to have on any EA growth dashboard.

-

(Let me know if you want me to PayPal you $2 or donate it wherever you wish.)

Comment author: Gleb_T  (EA Profile) 23 February 2016 09:32:56PM 0 points [-]

Here's the GWWC Twitter: https://twitter.com/givingwhatwecan

Here's the GiveWell Twitter: https://twitter.com/GiveWell

Here's the GiveWell FB: https://www.facebook.com/GiveWell.org/?fref=ts

Here's the link to the EA local newsletter: https://eahub.org/groups/resources/newsletter

Donate to Intentional Insights: http://intentionalinsights.org/view/donate

Comment author: ericyu3 23 February 2016 09:51:14PM 3 points [-]

Emailed you about it - I'm interested. Is anyone else already planning on doing this?

Comment author: Peter_Hurford  (EA Profile) 24 February 2016 12:52:27AM 0 points [-]

Replied. You're the first person to offer to do the analysis.

Comment author: Stefan_Schubert 22 February 2016 10:29:26AM 2 points [-]

Good idea. I'd also be interested in data on how many meetups and talks EA give in different cities and what impact that has. If more meetups and talks in a city translate into significantly more growth that would seem to be a good reason to put more resources into such activities.

Comment author: Peter_Hurford  (EA Profile) 23 February 2016 07:58:57PM 0 points [-]

That seems pretty plausible to me but I don't know how to collect that data. Do you have any ideas?

Comment author: Tom_Ash  (EA Profile) 24 February 2016 07:30:24PM 1 point [-]

LEAN is capturing it in our annual questionnaire for group organizers.

Comment author: Stefan_Schubert 24 February 2016 07:37:54PM *  0 points [-]

Great! What patterns have you spotted so far?

Comment author: Tom_Ash  (EA Profile) 02 March 2016 09:10:11PM 1 point [-]

We'll share the full results soon, including all non-confidential raw data. :)

Comment author: Stefan_Schubert 24 February 2016 10:28:01AM *  0 points [-]

One approach is to run an experiment. You choose a number of similar cities and then randomly increase the number of meetups and talks in one of them. Then you measure, e.g. whether this translates into more members in the local EA FB group (provided such a group exists), more people at physical meetings, etc. (I'm sure there are other ways to measure this; that's just off the top of my head.)

Such an experiment could also be used to test the much-discussed idea of whether fast growth is risky. If it turns out that the faster-growing chapters cause problems in some way, that might be a reason to think that the EA movement should not grow fast, or at least that it should not try to grow fast using the method used in the experiment. If, on the other hand, they don't cause any problems, that is a reason for the movement to try to grow faster. (I favour faster growth, I should add.)

Comment author: DavidNash 22 February 2016 12:01:51AM 1 point [-]

On Meetup you can get data on how many members have joined, how many members have looked at the page in the last three months and how many have said they are going to events.

Comment author: Peter_Hurford  (EA Profile) 23 February 2016 07:58:34PM 0 points [-]

Do you know if it's possible to get that data across all EA meetups?

Comment author: Tom_Ash  (EA Profile) 24 February 2016 07:30:51PM 1 point [-]

LEAN is capturing it in our annual questionnaire for group organizers, and also our monitoring and evaluation of meetup.com accounts in particular (many of which we provide).

Comment author: markobalogh 26 February 2016 08:31:05AM 0 points [-]

Although certainly not the comprehensive or rigorous information you are looking for, google trends can provide some interesting insights on a broad scale. For example, while effective altruism is showing strong growth in terms of search frequency, it is still searched for 7 times less frequently than Peter Singer: http://www.google.com/trends/explore?hl=en-US&q=Effective+altruism,+/m/05xnb&cmpt=q&tz=Etc/GMT%2B8&tz=Etc/GMT%2B8&content=1

Comment author: turchin 23 February 2016 02:22:03PM 0 points [-]

May be you could look on translation of main articles into different languages?

Comment author: Peter_Hurford  (EA Profile) 23 February 2016 08:06:01PM 0 points [-]

Like the rate at which that happens? What kind of data would you envision collecting?