Comment author: Elizabeth 26 April 2017 11:40:14PM 1 point [-]

i can see it clearly now, not sure if I was inattentive or something went wrong the first time I loaded the page.

Comment author: Jeff_Kaufman 27 April 2017 07:02:00PM 2 points [-]

if it happens again, I'd love a screenshot so I can debug

Comment author: Elizabeth 24 April 2017 01:46:10PM *  3 points [-]

Fyi, original comment was deleted and it now looks like you're criticizing a reasonable post, at least on mobile.

Comment author: Jeff_Kaufman 26 April 2017 08:46:19PM 2 points [-]

it now looks like you're criticizing a reasonable post, at least on mobile

When I look at this on mobile I see: https://goo.gl/photos/uzum9yU8YWceuUSg6

This doesn't look confusing to me, but does it to you? Or do you see something else?

(If the layout makes it look like replies to deleted comments are replies to the post, that's a problem we should and can fix.)

Comment author: Jeff_Kaufman 08 February 2017 02:46:14AM 1 point [-]

"I reused the diet questions in my plan from MFA 2013 study on leafleting"

In my view, this study asked way too much. When you try to ask too much detail people drop out. Additionally, it asks about things like diet change, but to pick up on changes we should be comparing the experimental and control groups, not comparing one group with its (reported) earlier self.

What I'd like to see is just "do you eat meat" along with a few distractor questions:

  1. Are you religious?
  2. Is English your native language?
  3. Do you eat meat?
  4. Do you own a car?

Yes, we'd like to know way more detail than this, and in practice people are weird about how they use "meat", but the main issue here is getting enough responses to be able to see any difference at all between the two groups.

Comment author: Jeff_Kaufman 08 February 2017 02:36:55AM 1 point [-]

"A question to determine if they were leafleted or not, without directly asking."

People who were leafleted but ignored it and don't remember enough to answer this one accurately is a problem here.

What would you think of: at a college that allows students to mass pigeonhole directly, put experiment leaflets in odd mailboxes and control ones in even boxes. Then later put surveys in the boxes, with different links for odd and even boxes.

Instead of having the links be example.com/a and example.com/b it would be better for them all to look like example.com/gwfr so people don't know what's going on. You could generate two piles of follow-up links and use one for the odd boxes and the other for even. QR codes might be good to add so people have the option not to type.

Comment author: Ben_West  (EA Profile) 30 January 2017 04:47:13PM *  3 points [-]

Thanks for the feedback!

My choice of minimum wage was originally motivated by Peter Singer's "if it is in our power to prevent something very bad from happening, without thereby sacrificing anything else morally significant, we ought, morally, to do it." Going below minimum wage seemed like as good a threshold as any for "morally significant" sacrifice.

I no longer think that's true, partly because of the various welfare benefits people on minimum wage the US get (like you point out), and also partly because I no longer think cutting expenses is the best way for me to improve the world.

Anyway, that long-winded introduction is just to say that, when I deviate from some general understanding of "living on minimum wage", it's not clear to me whether that's a bad thing.

On the other hand, people's normal understanding of "living on minimum wage" includes things like "paying for housing and food".

Yeah, that's a fair point. Although at least in Wisconsin I think an individual on minimum-wage would qualify for food stamps, and possibly a rent subsidy as well?

Do you mean that you don't count the money you pay in taxes as part of your spending

Yeah, that's what I meant.

Comment author: Jeff_Kaufman 31 January 2017 04:29:00PM 2 points [-]

an individual on minimum-wage would qualify for food stamps, and possibly a rent subsidy as well?

My (MA) understanding is that food stamps are relatively easy to get, and most people with low enough income can get them, but housing subsidy is very hard to get, with a queue of several years.

Which is another thing that makes this tricky: there are a lot of benefits that people are technically entitled to but often don't get in practice, and often take up a lot of time to get. So if you look at the CBO's "Effective Marginal Tax Rates for Low- and Moderate-Income Workers in 2016" pdf (figure 1) you can see that there are significant benefits at the low end, but lots of people don't get them.

Comment author: Jeff_Kaufman 29 January 2017 06:58:57PM 6 points [-]

Thanks for writing this up!

I don’t deduct any of my food or housing costs even though I need those to live

This seems like a big one. Your biggest expenditures are necessities: food and housing. You might be ok with cheaper housing and food, but certainly bringing your expenses to $0 in these categories would be extremely PWPF. On the other hand, people's normal understanding of "living on minimum wage" includes things like "paying for housing and food". So deducting the cost of minimally acceptable housing and food as PFPW wouldn't make much sense.

I don't track taxes

Do you mean that you don't count the money you pay in taxes as part of your spending, or something else? The former makes sense to me: someone living on minimum wage would probably pay negative tax on net, via EITC.

Comment author: erikaalonso 13 January 2017 12:38:41AM *  21 points [-]

Hi everyone! I’m here to formally respond to Sarah’s article, on behalf of ACE. It’s difficult to determine where the response should go, as it seems there are many discussions, and reposting appears to be discouraged. I’ve decided to post here on the EA forum (as it tends to be the central meeting place for EAs), and will try to direct people from other places to this longer response.

Firstly, I’d like to clarify why we have not inserted ourselves into the discussion happening in multiple Facebook groups and fora. We have recently implemented a formal social media policy which encourages ACE staff to respond to comments about our work with great consideration, and in a way that accurately reflects our views (as opposed to those of one staff member). We are aware that this might come across as “radio silence” or lack of concern for the criticism at hand—but that is not the case. Whenever there are legitimate critiques about our work, we take it very seriously. When there are accusations of intent to deceive, we do not take them lightly. The last thing we want to do is respond in haste only to realize that we had not given the criticism enough consideration. We also want to allow the community to discuss amongst themselves prior to posting a response. This is not only to encourage discussion amongst individual members of the community, but also so that we can prioritize responding to the concerns shared by the greatest number of community members.

It is clear to us now that we have failed to adequately communicate the uncertainty surrounding the outcomes of our leafleting intervention report. We absolutely disagree with claims of intentional deception and the characterization of our staff as acting in bad-faith—we have never tried to hide our uncertainty about the existing leafleting research report, and as others have pointed out, it is clearly stated throughout the site where leafleting is mentioned. However, our reasoning that these disclaimers would be obvious was based on the assumption that those interested in the report would read it in its entirety. After reading the responses to this article, it’s obvious that we have not made these disclaimers as apparent as they should be. We have added a longer disclaimer to the top of our leafleting report page, expressing our current thoughts and noting that we will update the report sometime in 2017.

In addition, we have decided to remove the impact calculator (a tool which included an ability to enter donations directed to leafleting and receive estimates of high and low bounds of animals spared) from our website entirely until we feel more confident that it is not misleading to those unfamiliar with cost effectiveness calculations and/or an understanding of how the low/best/high error bounds exemplify the uncertainty regarding those numbers. It is not typical for us to remove content from the site, but we intend to operate with abundant caution. This change seems to be the best option, given that people believe we are being intentionally deceptive in keeping them online.

Finally, leadership at ACE all agree it has been too long since we have updated our Mistakes page, so we have added new entries concerning issues we have reflected upon as an organization.

We also notice that there is concern among the community that our recommendations are suspect due to the weak evidence supporting our cost-effectiveness estimates of leafleting. The focus on leafleting for this criticism is confusing to us, as our cost-effectiveness estimates address many interventions, not only leafleting, and the evidence for leafleting is not much weaker than other evidence available about animal advocacy interventions. On top of that, cost-effectiveness estimates are only a factor in one of the seven criteria used in our evaluation process. In most cases, we don’t think that they have changed the outcome of our evaluation decisions. While we haven’t come up with a solution for clarifying this point, we always welcome and are appreciative of constructive feedback.

We are committed to honesty, and are disappointed that the content we've published on the website concerning leafleting has caused so much confusion as to lead anyone to believe we are intentionally deceiving our supporters for profit. On a personal note, I’m devastated to hear that our error in communication has led to the character assassination not only of ACE, but of the people who comprise the organization—some of the hardest working, well-intentioned people I’ve ever worked with.

Finally, I would like everyone to know that we sincerely appreciate the constructive feedback we receive from people within and beyond the EA movement.

*Edited to add links

Comment author: Jeff_Kaufman 14 January 2017 04:30:27PM 1 point [-]

We have recently implemented a formal social media policy which encourages ACE staff to respond to comments about our work with great consideration, and in a way that accurately reflects our views (as opposed to those of one staff member).

Is this policy available anywhere? Looking on your site I'm finding only a different Social Media Policy that looks like maybe it's intended for people outside ACE considering posting on ACE's fb wall?

Comment author: Peter_Hurford  (EA Profile) 05 January 2017 07:25:49PM 0 points [-]

That sounds pretty awesome, who do you think would be a good person to reach out to when I'm ready?

Comment author: Jeff_Kaufman 06 January 2017 09:07:28PM 0 points [-]

Ben Kuhn maybe?

Comment author: Peter_Hurford  (EA Profile) 04 January 2017 07:43:12PM 1 point [-]

Thanks Jeff!

-

1) ... I think the chance of failure could be significantly higher.

Possibly, but they are already starting to operate in the country in question, and my understanding is that's been going pretty well. My impression is that they're much more competent than Safaricom. My inside view is much higher than 50%, and getting down to 50% was a discount from there.

Sounds like you definitely have inside info that I don't have, so for now I'd have to rely on my outside view, but I can work to acquire that inside info if I look into this more.

-

2) ... I expect future roll-outs will take place in countries with higher base consumption

I'm confused. I was trying to talk about the counterfactual for a specific very poor country if Wave were not working there. So if future mobile money rollouts by other organizations happen first in countries with higher base consumption then that increases the counterfactual impact of Wave choosing to come into a country with very low consumption.

I don't know what country Wave is looking at or how they are doing what they do because you have inside info that I don't have. If it has consumption comparable to Kenya than my point is invalid. I just was concerned that it wouldn't.

-

3) ... I expect them to continue at AMF levels (or greater) for at least a few more years

See http://www.jefftk.com/p/leaving-google-joining-wave#fb-835897806972_835943804792

Cool. Sounds like this isn't a disagreement between us then.

-

4) ... I really don't know how many staff years it would take

That, combined with estimating marginal impact, makes this pretty awkward. I figure something like 40 person years?

Agreed that it is pretty awkward to estimate. I modified my model to use some of your inputs -- such as a 40% chance of 1-10M subscribers and a 10% chance of >10M subscribers and 40 person years -- and it comes out to $383/hr (95%: $145/hr to $834/hr). The new mean is still in my old 95% interval which is about the best I can hope for with this level of uncertainty.

-

5) ... I'm confused about why GiveDirectly is stated to be 5x more cost-effective than AMF

This comes from cell F31 of the "Results" tab. I haven't put time into understanding how that's calculated, but it looked like the relevant bottom line number.

Oh, I see that now. I suppose this is a question for GiveWell and not you. I'll ask them.

Comment author: Jeff_Kaufman 05 January 2017 06:29:29PM 1 point [-]

Sounds like you definitely have inside info that I don't have, so for now I'd have to rely on my outside view, but I can work to acquire that inside info if I look into this more.

If you're interested in working for Wave, or are advising other people on whether it's a good idea for them, I could imagine they'd be quite interested in talking to you!

if it has consumption comparable to Kenya than my point is invalid. I just was concerned that it wouldn't.

It's poorer than Kenya.

Comment author: Peter_Hurford  (EA Profile) 04 January 2017 06:13:00PM 1 point [-]

I spent about two hours looking at this in further depth and made an initial stab at modeling out the impact. I estimate an effectiveness of $200/hr (95% interval: $50/hr to $511/hr), not taking into account the value of donating the salary earned from working at Wave.

Some places where I notice we disagree or I am confused:

1.) I disagree with you here (footnote 1) that there is a 50% chance of failure (or success). I think the chance of failure could be significantly higher. From https://en.wikipedia.org/wiki/M-Pesa: M-Pesa expanded to Kenya (>10M subscribers), Tanzania (5M), South Africa (100K in a year, 1M in five years), India (???), Mozambique (???), and Lesotho (???).

Also, a 2016 Vodaphone press-release suggested M-Pesa seems to have 25M customers worldwide after 10 years of effort.

Based on this, I model that a Kenya-level success (>10M subscribers) thus looks like it would have a less than 1/10 chance and a South Africa-level success (1-10M subscribers) looks like it would have a ~3/10 chance. However, I think this success figure could be lower due to diminishing marginal returns since M-Pesa has already plucked low hanging fruit. It's possible better technology could increase this chance. I'd have to know more specifically about what problems M-Pesa runs into and how these are addressed.

-

2.) I think your estimate that getting M-Pesa a year earlier is only 66x worse than getting a $288 transfer from GiveDirectly is an overestimate because I expect future roll-outs will take place in countries with higher base consumption. However, as you point out, that estimate is also already an underestimate due to misunderstanding the study. I don't know how to correct for this either way, so I used the 66x number literally in my calculation.

-

3.) I disagree with you here (footnote 1a) that marginal ETG donations are at GiveDirectly levels of cost-effectiveness. I expect them to continue at AMF levels (or greater) for at least a few more years, for reasons OpenPhil mentioned and Carl mentioned. I did an AMF-adjustment in my model for this reason.

-

4.) I really don't know how many staff years it would take to either complete a roll-out or know that it's not going to happen and this is an important part of the model. I currently guess 5-10 full-time staff for 2-5 years, or 10-50 total staff years. This does not count field agents or other hired locals. I couldn't find any information on M-Pesa's total staff count anywhere. I note that Wave has at least 44 staff (from counting faces on the about page), but I don't know if they're all full-time or all focused on expanding cash transfers.

-

5.) I'm confused about why GiveDirectly is stated to be 5x more cost-effective than AMF when from GiveWell's cost-effectiveness estimate, GiveDirectly has a median of $7702 per life saved, ranging from $2200 to $16000, excluding outliers. AMF has a median of $3282 per life saved, ranging from $2200 to $4800, excluding outliers. Together, this implies a comparison centered around 2.35x but ranging from 1x to 3.3x. Maybe I misread the sheet -- I haven't invested that much time in making sure I fully understand it yet.

Comment author: Jeff_Kaufman 04 January 2017 07:23:54PM 1 point [-]

1) ... I think the chance of failure could be significantly higher.

Possibly, but they are already starting to operate in the country in question, and my understanding is that's been going pretty well. My impression is that they're much more competent than Safaricom. My inside view is much higher than 50%, and getting down to 50% was a discount from there.

2) ... I expect future roll-outs will take place in countries with higher base consumption

I'm confused. I was trying to talk about the counterfactual for a specific very poor country if Wave were not working there. So if future mobile money rollouts by other organizations happen first in countries with higher base consumption then that increases the counterfactual impact of Wave choosing to come into a country with very low consumption.

3) ... I expect them to continue at AMF levels (or greater) for at least a few more years

See http://www.jefftk.com/p/leaving-google-joining-wave#fb-835897806972_835943804792

4) ... I really don't know how many staff years it would take

That, combined with estimating marginal impact, makes this pretty awkward. I figure something like 40 person years?

5) ... I'm confused about why GiveDirectly is stated to be 5x more cost-effective than AMF

This comes from cell F31 of the "Results" tab. I haven't put time into understanding how that's calculated, but it looked like the relevant bottom line number.

View more: Next