joshjacobson comments on EA Survey 2017 Series: Cause Area Preferences - Effective Altruism Forum

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (43)

You are viewing a single comment's thread.

Comment author: joshjacobson  (EA Profile) 11 September 2017 03:03:53PM -1 points [-]

I think it's quite misleading to present p-values and claim that results are or aren't 'statistically significant', without also presenting the disclaimer that this is very far from a random sample, and therefore these statistical results should be interpreted with significant skepticism.

Comment author: Peter_Hurford  (EA Profile) 12 September 2017 03:02:56AM 2 points [-]

This is covered in detail in the methodology section. We try not to talk about statistical significance much, we try to belabor that these are EAs "in our sample" and not necessarily EAs overall, and we try to meticulously benchmark how representative our sample is to the best of our abilities.

I agree some skepticism is warranted, but not sure if the skepticism should be so significant as to be "quite misleading"... I think you'd have to back up your claim on that. Could be a good conversation to take to the methodology section.

Comment author: joshjacobson  (EA Profile) 12 September 2017 02:59:16PM 0 points [-]

I haven't looked there yet, so I'm flagging that my comment was not considering the full context.

(I think that the end links didn't come up on mobile for me, but it could also have been an oversight on my part that there was supporting documentation, specifically labelled methodology.)