saulius comments on A lesson from an EA weekend in London: pairing people up to talk 1 on 1 for 30 mins seems to be very useful - Effective Altruism Forum

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (13)

You are viewing a single comment's thread. Show more comments above.

Comment author: saulius  (EA Profile) 12 June 2018 02:14:41PM *  1 point [-]

1) >"The numbers on how useful things are seem quite low to me..."

On the scale 1 was "Useless" and 10 was "Life-transforming". But just before asking for feedback, I made a change in the slides and added this meaning to the ratings of the events:

"3 - £100, 5 - £1,000, 8 - £10,000, 10 - £100,000 (e. g. career change)"

I explained it to people as well. This was... not smart. Because of this, some respondents gave low scores to all the events. E.g. someone said that the weekend was "Far more valuable (10-30x the counterfactual)" but did not gave any event a rating that is higher than 4. Others ignored the point and gave high ratings for all events.

That's why I weighted and normalised the ratings. If someone said that the weekend was "Vastly more valuable (>30x counterfactual)", I multiplied all their ratings by a constant so that their highest rating would be 10. If they rated the weekend as "Far more valuable (10-30x the counterfactual)", I multiplied all their ratings so that the highest rating would be 9. 8 for "Much more valuable", 7 for "Somewhat more valuable", and 6 for "About as valuable".

Comment author: MichaelPlant 12 June 2018 03:03:06PM 0 points [-]

Yeah, I thought the ends of the scales might have been more extreme than we'd normally use. It's probably quite hard to get people to sensibly answer unfamiliar, tricky questions.