21

Effects of major events on EA activity

Peter proposed using time series of EA activity (Google searches, GWWC pledges, etc.) to see whether major events like EA Global and GWWC’s pledge drive are associated with spikes of interest in EA. I looked at the data, and found many of the correlations that you would expect (GWWC does get more pledges during their pledge drive), but also a few surprises (e.g. EA Global has few obvious effects).

Although there wasn’t enough data to get a particularly detailed picture of EA growth, many spikes in activity coincided with major events, suggesting that the events caused the increases. To see how different events corresponded to spikes in activity (or didn’t), see the annotated plots below.

Data sources

  • Facebook posts weighted by comments, likes, and shares (from sociograph.io)

  • GiveWell website traffic and mailing list sign-ups (from Tyler Heishman at GiveWell)

  • GWWC pledges (from https://www.givingwhatwecan.org/dashboard)

  • Google search interest in “effective altruism” (from Google Trends)

Analysis

Dates of major EA events:


  • Effective Altruism Summit 2014: 8/2-8/3, 2014

  • GWWC pledge drive 2014–2015: a few weeks around 1/1/2015

  • GWWC pledge drive 2015–2016: a few weeks around 1/1/2016

  • EA Global 2015: 7/31-8/2, 2015 (SF), 8/14-8/16 (Melbourne), 8/28-8/30 (Oxford)

  • Doing Good Better published: 7/28/2015 (around the same time as EA Global)

  • The Most Good You Can Do published: 3/15/2015 (published on Amazon 4/7/2015)

  • Peter Singer’s TED talk: March 2013

  • GiveWell top charities announced (2014–2015): 12/1/2014

  • GiveWell top charities announced (2015–2016): 12/18/2015

 

For each metric, I collected data on cumulative activity over time, filled in missing data points using linear interpolation, downsampled into 1-week bins, calculated week-over-week changes in cumulative activity, and normalized the weekly rates to the activity’s average rate over all weeks. I plotted the following time series:


  • Facebook posts

  • Total number of times the Facebook posts were shared

    • Includes shares that occur after the week ends

  • New GWWC members

  • New GiveWell mailing list members

  • Google search interest

Plots for the most informative metrics can be viewed with Tableau—click on the images below for links. All metrics are normalized to their mean value over time; for example, a value of 1.2 for Facebook posts means there were ~20% more posts that week compared to the average week in the whole period since February 2014.

Facebook activity (orange = posts, brown = posts weighted by shares):


 

GiveWell mailing list signups (red) and GWWC pledges (green):

 

 

Google searches for “effective altruism”:

 

Some metrics were left out of the plots, either due to incomplete data (GiveWell website traffic) or lack of informativeness (the rate of Facebook likes is highly correlated with the rate of posts).


GWWC and GiveWell have major spikes in pledges and mailing list signups during the holiday season. However, GWWC did not have a spike in the 2013–2014 season, when they did not have a drive—this strongly suggests that their pledge drives are causing the spikes in pledge rates. GiveWell’s spikes occur earlier, probably because they announce their top charities before GWWC’s pledge drives start. Facebook and Google search activity don’t spike at the same time, and surprisingly, GWWC’s monthly traffic (not plotted) isn’t significantly higher during their pledge drives.


Other interesting trends:


  • GWWC gains members at a fairly constant rate outside of pledge drives

  • The number of Facebook shares is more informative than the number of posts, since the rate of posting varies very little

  • EA Global and Doing Good Better don’t have as large of an effect as you might expect. There seems to be a spike in highly-shared Facebook posts around that time, but not in the overall number of posts.

  • Google search interest seems to depend mainly on media coverage. EA Global/Doing Good Better had a large effect, but the 2014 Effective Altruism Summit did not. Unfortunately, since EA Global occurred at the same time as the book’s publication, it’s not clear which event caused the spike.

  • Google search interest rose quickly until mid-2015, but is fairly stable now and has stayed fairly constant since (although only a limited amount of time has passed). Three of the largest spikes occurred when:

    • Peter Singer gave his TED talk

    • EA Global was held and Doing Good Better was published (The Most Good You Can Do didn’t have an obvious effect)

    • GiveWell published their updated list of top charities for 2015 (no similar spike happened in 2014)

  • Facebook posts and shares grew quickly between 2014 and mid-2015, but are growing much more slowly now

Total activity

To give an idea of how significant the spikes are, here are the approximate average weekly rates of each activity:


Google Trends interest since 2013

~40% of the December 2015 peak

FB group posts

30.7

FB group likes

306.2

FB group comments

186.1

FB group shares

15.8

GWWC pledges

5.0

GiveWell traffic

8160.4

GiveWell mailing list sign-ups

11.8

Conclusions

Overall trends

The single most surprising pattern was that different types of activity seem to be driven by very specific things. There’s no event that causes a spike in all metrics, and only EA Global and GiveWell’s 2015 top charities announcement are associated with a spike in more than one.


Another surprising trend is that none of the activity metrics have grown much since mid-2015. This could mean that EA’s growth is slowing down. More optimistically, this could simply be because we don’t have enough data yet to detect for growth. For GWWC membership, arguably the most important metric, it’s still too early to tell whether people have been joining at a higher rate this year.

“Awareness” vs. “membership”

Google Trends seems to be the best indicator of how much publicity effective altruism is getting and how many people are aware of EA, since the level of interest spikes strongly when books, talks, and news articles are published.


On the other hand, the total number of GWWC pledges and GiveWell mailing list subscriptions seems like a better measure of “how many effective altruists there are”. (The most concerted effort to find as many self-identified EAs as possible is .impact’s annual EA Survey, but this only lets us see the impact of events months after the fact, and it’s difficult for an annual survey to isolate the effects of individual events throughout the year.)


However, this doesn’t necessarily mean that having more events to boost GWWC/GiveWell affiliation would increase the number of EAs. For instance, people who pledged during GWWC’s pledge drives might have been planning to join anyway at a different time—see Linchuan Zhang’s post for a more detailed discussion of this issue.


The rate of GWWC pledges outside of pledge drives and the level of Google search interest grew at roughly the same rate, both doubling from 2014 to 2015. If this trend continues, it would suggest that growth is reasonably balanced between (1) public visibility and (2) attracting people who agree with EA enough to donate 10% of their income.

What other kind of data would be the most useful?

The three main types of metrics—Facebook posts, organization membership, and interest on Google Trends—gave distinct and complementary information about EA’s growth, so having another type of metric would likely be more informative than additional web traffic or mailing list signup statistics. Ideally, we would have detailed information on when, where, and how much people are donating, but this data will be much harder to collect.


 

Comments (6)

Comment author: aliwoodman 29 April 2016 05:22:04PM 6 points [-]

Thanks for the writeup! Really interesting to get a better picture of this.

Just to add some extra info about the effects of EA Global and 'Doing Good Better' and Singer's TED talk on GWWC member growth. It's true EA Global last year didn't lead to new members straight away, but 2 people joined later in the year citing EA Global as a way they first heard about Giving What We Can. In addition, after following up with participants individually, 7 took the pledge.

'Doing Good Better' has had a much bigger effect. 72 of the members who have signed up since July 2015 (11%) mentioned 'Doing Good Better' in response to the pledge form question of 'How did you first heard about Giving What We Can?', making up 6.5% of the overall responses to the question (people can select multiple options). The distribution of join dates for these members is fairly even, with higher numbers in January and February.

In total Peter Singer's TED talk accounts for 10% of the responses to the 'How did you first hear..?' question. Again the proportion of members influenced by this seems pretty steady over time.

(In case unclear, I work at Giving What We Can :) )

Comment author: Habryka 29 April 2016 08:55:03PM 3 points [-]

Excellent, thank you a lot for doing this analysis!

Would it be possible to extract the concrete average growth rates and post them in this post?

E.g.: 5 members/month for GWWC, 3 Signups/day, etc.

Comment author: Linch 29 April 2016 09:45:13PM 2 points [-]

This is relatively easy to do with GWWC:

https://www.givingwhatwecan.org/dashboard

Eyeballing the dashboard, GWWC has 50-60 members/month outside the pledge drives in 2015 and 2016, or about 11-14 members/week, and about 30ish/month in 2014, or about 7/week.

Comment author: SoerenMind  (EA Profile) 30 April 2016 05:35:56PM *  2 points [-]

Just to add: The monthly signups for the EA newsletter at effectivealtruism.org look roughly constant at 250/month since October 2015. There's a slight spike of 320 in December.

Before October there was no newsletter but you could subscribe anyway. The rate was ~180/month from April to July and jumped up to 300 in August where EAG and DGB happened. Then it settled around 250/month. There's enough variation for there to be an slight trend though that we can't spot though.

Comment author: zdgroff 02 May 2016 01:16:22AM 1 point [-]

Re: Limited increase following EA Global - I would guess that EAG is most useful for consolidating people who already know about EA and linking them to each other in ways that increase productivity.

Re: different trends being caused by very specific things - I would guess EA as a movement only has a very small group of people who are fully aware of EA, interested in the various domains and ways of thinking. It has many more people who are involved in one sense or another - giving to effective charities, giving a large sum of their income, choosing to work based on effective cause selection, etc. These trends seem to suggest that EA is largely not sold as a bundle.

Comment author: Linch 29 April 2016 10:17:09PM 1 point [-]

Great post Eric!

Is there a way to track web traffic outside of Google Searches? For example, how often "effective altruism" or "giving what we can", etc. is mentioned in the media, blog posts, etc.