12

Is EA Growing? Some EA Growth Metrics for 2017

This post was co-authored by Peter Hurford and Joey Savoie.

 

The EA Survey team at Rethink Charity (including myself) recently released initial data from the 2017 EA Survey and will have more to follow it up. KBog made a comment on the EA Forum noticing that the 2015 EA Survey had 2532 participants, whereas the 2017 EA Survey only had 1837 participants. Does this mean that EA is declining in growth?

It’s hard to say, and the EA Survey team will have more analysis on this, looking deeper at what the data in the EA Survey tells us about EA membership growth and churn, if anything. However, Joey Savoie (unrelated to the EA Survey team) and I (Peter Hurford) were curious to look a bit more at other metrics of potential growth to get a clearer picture. Prior analysis on this has been done by Vipul Naik at the end of 2016 and by Eric Yu at the beginning of 2016 (in a post that I sponsored the creation of).

Joey and I thought it would be a good time to try to re-check the data and see if things have changed. We compiled the following data from various sources to look at EA growth rates.

Type of data

Sep 2014 to Aug 2015

Sep 2015 to Aug 2016

Sep 2016 to Aug 2017

The EA Facebook group (via sociograph.io)[1]

1547 posts

471 authors

920 commenters

1924 reactors[2]

1374 posts

521 authors

1055 commenters

2710 reactors

696 posts[3]

400 authors

1089 commenters

2872 reactors

New Giving What We Can pledgers

618

770

1150

Number of 80,000 Hours impact-adjusted significant career changes

184.8

631.3

1202

Google interest in “effective altruism” (relative scoring)[4]

16.94

28.4

38.2

EA wikipedia data (desktop pageviews)[5]

51,100

70,500

77,800

New EA Newsletter sign-ups

Didn’t really exist[6]

11,167

35,554

EA Forum pageviews

309,961

345,202

390,463

New EA Reddit subscribers[7,8]

308

869

1484

 

Type of data

January 2014 to Dec 2014

January 2015 to December 2015

January 2016 to December 2016

Non-OpenPhil GiveWell donations

$13.3M

$40M

$30-40M (math)

OpenPhil GiveWell donations[9]

$8M

$70M

$50M

Total OpenPhil donations

$30.2M

 

$34.2M

$126.4M

OpenPhil + non-OpenPhil GiveWell donations

$21.3M

$110M

~$80-90M

Total non-OpenPhil donors

9044

14,287

No data available[10]

Total recorded money actually donated (not pledges) from Giving What We Can members

$7.1M

$7.0M[11]

No data available

80,000 Hours Newsletter Subscribers

262

23,000

76,000

 

Type of data

EA Survey 2014

EA Survey 2015

EA Survey 2017

EA Survey respondents

813

2532

1837

Total amount of recorded donations in EA Survey

$5.2M

$6.8M

$9.9M[12]

 

Additional Facebook Data

 As an admin of the EA Facebook group, I was also able to access the built-in statistics, but they only go back 60 days. Right now, as of 30 August 2017, the EA FB group has about 8629 “active users” in the last 60 days as defined by Facebook (user has viewed, posted, commented, or reacted in the group). The last 60 days also saw 99 posts, 2073 comments, and 7042 reactions. 339 members were added in the past 60 days, to a total of 13,407 members right now. Comparing the past sixty days to the past 28 days shows roughly linear growth, but I would not expect to be able to see a trend in only two months of data, even if EA was growing very quickly.

This data is not very useful right now since it only goes back 60 days, but it might be valuable as a time capsule, since a year from now we could re-do this post and compare the numbers for the group as of 2018 to these archived numbers.

 

Commentary 

Overall, it’s hard to get a good sense of a trend from only two to three data points for each trend. It looks like EA is growing somewhat and that the growth rate is somewhat linear, but it varies depending on the source and I would definitely want to see more data to be sure. A lot of the growth rate can vary depending on how much marginal resources organizations like the Center of Effective Altruism put into intentionally growing a particular source (e.g., the EA Newsletter) versus letting a source grow organically (e.g., the EA Forum). 

I don’t really know what kind of growth rate I was expecting prior to seeing this data, so I can’t say if this does or doesn’t support my hypotheses about movement growth. Joey and I are trying to hold off from having firm additional opinions until we can look at the data more. I think it would be ideal to create concrete predictions around where these numbers will be within one year or so (e.g., see herehere, and here). We certainly invite discussion and predictions on this.

 

Acknowledgements 

This post was co-authored by Peter Hurford and Joey Savoie. Thanks to Vipul Naik and Issa Rice for putting together some of the source data used in this report and referring us to the correct places for getting EA Forum and EA Wikipedia pageview data, and thanks Pascal Zimmer for supplying data on the EA Newsletter. Thanks to Kerry Vaughan for providing data on pageviews for the EA Forum in 2017, thanks to Rob Wiblin for some 80,000 Hours statistics, and thanks to Zeke Sherman for pointing out Reddit statistics.

 

Endnotes 

[1]: Sociograph does offer to show member growth over time as a paid feature, which I got via a 14-day free trial, but the data looked incorrect and unusable. 

[2]: I believe “reactors” refer to the number of people who like, haha, love, etc. I am not sure if this includes shares, but I do not think that it does.

[3]: It’s worth noting that the EA Facebook group engages in reasonably heavy moderation of Facebook posts and probably (my intuition as a Facebook mod) rejects about five posts for every one post that reaches the page. Given that moderation may have changed over time, it’s not clear how much to read into this decline in the post count.

[4]: These numbers are not search volumes -- they’re the mean relative “score” for that year, relative to the search volume for the highest day between January 2004 and the end of August 2017.

[5]: See some more wiki data here and here.

[6]: The EA Newsletter was relaunched and first had regular content and marketing in October 2015.

[7]: Both r/EffectiveAltruism and r/smartgiving have been simultaneous EA subreddits since September 2012. r/smartgiving was the default EA subreddit until an intentional migration on 28 Feb 2016. From Sep 2016 - Aug 2017, r/effectivealtruism had +1484 subscribers (1068->2552) and r/smartgiving had +93 (1466->1559); from Sep 2015 - Aug 2016, +869 (199->1068) for r/effectivealtruism and +230 (1236->1466) for r/smartgiving; and from Sep 2014 - Aug 2015, +91 (108->199) for r/effectivealtruism and +308 (928->1236) for r/smartgiving. I will use r/smartgiving numbers for the 2014-2015 period and r/effectivealtruism numbers for all periods after that, to reflect the transition. Note that this growth will therefore involve some inherent double-counting as people who were subscribed on r/smartgiving re-subscribe on r/effectivealtruism. Pageviews for reddit were calculated via http://redditmetrics.com/.

[8]: Reddit admin stats go back a year for pageviews. Looking at this, there were ~65,350 pageviews between Sep 2016 and August 2017. We can keep this number in this comment as a time capsule and then if/when we do a Growth Metrics 2018 we can compare the pageviews to this number.

[9]: Money from the Open Philanthropy Project is counted for the year in which the grant is announced, which may be different from the year the grant is decided or the year the grant money is actually dispersed.

[10]: GiveWell has not yet published their Metrics Report for 2016 data.

[11]: These numbers come from different sources, so I don’t know if they are directly comparable. 

[12]: The EA Survey 2017 recorded donations from both 2015 and 2016. This is 2016 data only. There was $6.7M in donations recorded for 2015 in the 2017 EA Survey.

Comments (16)

Comment author: Robert_Wiblin 06 September 2017 07:18:40AM *  0 points [-]

You should really add non-'global health and development' OpenPhil grants as that's now almost as much as all the other donations here listed together (e.g. $105m since the start of 2017, vs $60m through all of 2016 and little before that). Easy to get all the data here: http://www.openphilanthropy.org/giving/grants .

Adding in that the growth rate looks much faster.

Comment author: Peter_Hurford  (EA Profile) 06 September 2017 11:40:16AM 0 points [-]

Good idea. I added that in.

Comment author: weeatquince  (EA Profile) 07 September 2017 11:33:52AM *  1 point [-]

Hi, In case helpful for considering the additional Facebook information, I have a bunch of data on EA social media presence to help me compare growth in London to other locations, including a lot of downloaded Sociograph data from 2016.

For example the EA Facebook group size over the last year:

03/06/2016 _ 10263

13/01/2017 _ 12070

10/06/2017 _ 12,953

Obviously you'd expect these things to grow as people join then do not leave (but might ignore it), even if the movement was shrinking.

Comment author: Robert_Wiblin 07 September 2017 02:08:24AM 1 point [-]

Regarding the rate of EA Facebook group posts - a year or two ago there was an effort to introduce moderation to slow the rate of low quality posts (often from entirely new people) that added noise and reflected poorly on the group. The lower engagement there seems to be a sign of success and professionalisation rather than slow movement growth.

Comment author: Peter_Hurford  (EA Profile) 07 September 2017 03:04:05AM 0 points [-]

Definitely. I note that in endnote #3, but it definitely should be pointed out. The heavier moderation started in December 2015.

Comment author: Robert_Wiblin 07 September 2017 09:32:30PM 0 points [-]

Sorry I missed the endnote. I think in that case, where the change is most naturally interpreted as neutral or positive, it's worth giving it a stronger caveat than a footnote, for people just skimming the numbers. :)

Comment author: Zeke_Sherman 06 September 2017 02:36:39PM *  1 point [-]

The effective altruism subreddit is growing in traffic: https://i.imgur.com/3BSLlgC.png (August figures are 2.5k and 9.5k)

The EA Wikipedia page is not changing much in pageviews: https://tools.wmflabs.org/pageviews/?project=en.wikipedia.org&platform=all-access&agent=user&start=2015-07&end=2017-08&pages=Effective_altruism

Comment author: vipulnaik 06 September 2017 06:50:01PM 0 points [-]

The subreddit stats used to be public (or rather, moderators could choose to make them public) but that option was removed by Reddit a few months ago.

https://www.reddit.com/r/ModSupport/comments/6atvgi/upcoming_changes_view_counts_users_here_now_and/

I discussed Reddit stats a little bit in this article: https://www.wikihow.com/Understand-Your-Website-Traffic-Variation-with-Time

Comment author: Peter_Hurford  (EA Profile) 06 September 2017 03:04:35PM 0 points [-]

Thanks. Good idea to include the subreddit. How do you get those stats? We can add traffic from the r/smartgiving subreddit too (r/effectivealtruism precursor) to go back prior to 2016.

I have EA Wikipedia pageview data in my post already. :)

Comment author: Zeke_Sherman 06 September 2017 04:31:17PM *  1 point [-]

You can find the stats by going to the right of the page in moderation tools and clicking "traffic stats". They only go back a year though. Redditmetrics.com should show you subscriber counts from before that, but not activity.

Comment author: Peter_Hurford  (EA Profile) 06 September 2017 05:38:30PM 1 point [-]

Thanks, I added the Reddit stats to the article!

Comment author: Robert_Wiblin 06 September 2017 07:27:09AM *  2 points [-]

I strongly recommend using the impact-adjusted plan change metric rather than the unadjusted one for 80,000 Hours. Those figures:

Sep 2014 to Aug 2015 - 184.8

Sep 2015 to Aug 2016 - 631.3

Sep 2016 to Aug 2017 - 1202

There's also our newsletter growth. New subscribers each year:

2014 - 262

2015 - 23,000

2016 - 76,000

2017 so far - 57,000.

Comment author: Peter_Hurford  (EA Profile) 06 September 2017 11:28:30AM 1 point [-]

Ok, done. Thanks. Impact-adjusted numbers are fairer, since that is what you are actually targeting, though there is some subjectivity in the impact adjusting process.

Comment author: Robert_Wiblin 06 September 2017 05:24:22PM 2 points [-]

Thanks. It's not so much that it's a matter of what we're targetting - I think some of these plan change are more than 100x more valuable than others. If you take the raw number of customers a company has, when some customers spend 100x more than others, the total number of customers could look good while the business is actually shrinking. Our raw plan change figures are dominated by the smallest, least important plan changes so could be quite misleading in future.

Comment author: MichaelPlant 06 September 2017 12:34:32PM 3 points [-]

Couldn't you put in both types of data so readers can draw their own conclusions?

Comment author: vipulnaik 06 September 2017 06:47:16PM 0 points [-]

I have been using PredictionBook for recording predictions related to GiveWell money moved; see http://effective-altruism.com/ea/xn/givewell_money_moved_in_2015_a_review_of_my/#predictions-for-2016 for links to the predictions. Unfortunately searching on PredictionBook itself does not turn up all the predictions because they use Google, which does not index all pages or at least doesn't surface them in search results.