In a 2013 TED talk Peter Singer claims
“It costs about 40,000 dollars to train a guide dog and train the recipient so that the guide dog can be an effective help to a blind person. It costs somewhere between 20 and 50 dollars to cure a blind person in a developing country if they have trachoma.”
Unfortunately, this claim is not accurate. To begin with, blindness from trachoma is irreversible so it's only possible to prevent blindness from trachoma, not to cure it. According to a GiveWell blog post, it does cost ~$20-60 to perform one trachoma surgery but “there can be a small improvement in vision following surgery”. According to their back-of-envelope calculation with some assumptions, 1 case of full-blown blindness is averted for every 6-20 successful surgeries. In any case, my point is that people who use this example to advertise GiveWell don't read what GiveWell has to say about it.
-------
EDIT (2017-05-16): Even though GiveWell haven't made such claim and may have a different opinion, one doctor (who has a much deeper understanding of these issues than me) commented that she "would be comfortable with saying that for about $100 we can prevent trachoma-induced blindness" and that Singer's claim was not as nearly as inaccurate as I made it seem.
-------
As of 2017-05-10, Giving What We Can also gives a similar example:
“In the developing world there are more than a million people suffering from trachoma-induced blindness and poor vision which could be helped by a safe eye operation, costing only about $100 and preventing 1-30 years of blindness and another 1-30 years of low vision, according to GiveWell.org”
They also do something EAs (including me) don’t do often enough — provide a source. The source is a GiveWell page which is published in 2009 and has a disclaimer
“The content on this page has not been recently updated. This content is likely to be no longer fully accurate, both with respect to the research it presents and with respect to what it implies about our views and positions.”
The page has the following text:
“We have not done thorough cost-effectiveness analysis of this program. Because such analysis is highly time-consuming - and because the results can vary significantly depending on details of the context - we generally do not provide cost-effectiveness analysis for an intervention unless we find what we consider to be a strong associated giving opportunity.
We provide some preliminary figures based on the Disease Control Priorities in Developing Countries report, which we previously used for cost-effectiveness estimates until we vetted its work in 2011, finding major errors that raised general concerns.
We have relatively little information about the likely impact of this program, so it's difficult to estimate the cost-effectiveness.”
[...]
“Using a simple conversion calculation, we estimate that $100 prevents 1-30 years of blindness and an additional 1-30 years of low vision when spent on surgeries (though insignificant benefits, in these terms, when spent on antibiotics). The source of the Disease Control Priorities in Developing Countries report's estimate is unclear and these figures should be taken with extreme caution.”
It seems unfair to just provide the numbers and skip all these disclaimers. Despite knowing about this uncertainty, sometimes I feel temptation to also omit disclaimers and just present the numbers to be more convincing. After all, the goal is very admirable - to help more people living in extreme poverty. But I believe that in the long run EA will achieve more if we are being totally honest and upfront about uncertainties and never take any shortcuts. Otherwise we might not be trusted the next time we have something to say. Furthermore, to influence the world we need our community to have a correct model of the world.
On the other hand, trachoma is a horrible disease. Just watch this excerpt:
tl;dw: eyelids turn inwards and eyelashes scrape the eyeball, causing intense pain on every blink. That scraping eventually causes blindness. People treat themselves by pulling out their eyelashes with tweezers. One woman said she does it every 2 weeks. Horrible.
If you worry about being convincing, you can talk about that and then honestly talk about uncertainty regarding numbers. Most people are scope insensitive anyway. Or you can talk about cataract surgery instead of trachoma because disclaimers in this page seem slightly less severe. Or just talk about your favorite charity and then add "imagine that suffering could be prevented so cheaply in our country, action would be taken urgently". But the main points of this post are
-
many of us were overstating the point that money goes further in poor countries
-
many of us don’t do enough fact checking, especially before making public claims
- many of us should communicate uncertainty better
-------
EDIT (2017-05-15):
Many people in the comments gave other reasons not to use the comparison but if you decide to use it anyway and want to quote GiveWell, you could also use figures from this Peter Singer's comment. Alternatively, you can use one of the other comparisons proposed by Ben Todd.
EDIT: this comment contains some mistakes
To begin with, I want to say that my goal is not to put blame on anyone but to change how we speak and act in the future.
I am looking at this table from the cited source (Loss of Vision and Hearing, DCP2). It’s 77% cure rate for trachoma that sometimes develops into blindness. Not 77% cure rate for blindness. At least that’s how I interpret it, I can’t be sure because the cited source of the figure in the DCP2’s table doesn’t even mention trachoma! From what I’ve read, sometimes recurrences happen so 77% cure rate from trachoma is much much more plausible. I'm afraid Toby Ord made the mistake of implying that curing trachoma = preventing blindness.
What is more, Toby Ord used the same DCP2 report that GiveWell used and GiveWell found major errors in it. To sum up very briefly:
Regarding Fred Hollows Foundation, please see GiveWell’s page about them and this blog post. In my eyes these discredit organization’s claim that it restores sight for $25.
In conclusion, without further research we have no basis for the claim that trachoma surgeries can prevent 400, or even 40 cases of blindness for $40,000. We simply don't know. I wish we did, I want to help those people in the video.
I think one thing that is happening is that we are too eager to believe any figures we find if they support an opinion we already hold. That severely worsens already existing problem of optimizer’s curse.
I also want to add that preventing 400 blindness cases for $40,000 (i.e. one case for $100) to me sounds much more effective than top GiveWell's charities. GiveWell seem to agree, see citations from this page
We tell the trachoma example and then advertise GiveWell, showing that GiveWell’s top and standout charities are not even related to blindness and no one in EA ever talks about blindness. So people probably assume that GiveWell’s recommended charities are much more effective than surgery that cures blindness for $100 but they are not.
Because GiveWell’s estimates for cataract surgeries are based on guesses, I think we shouldn’t use those figures in introductory EA talks as well. We can tell the disclaimers but the person who hears the example might skip them when retelling the thought experiment (out of desire to sound more convincing). And then the same will happen.
The mention of the specific errors found in DCP2 estimates of de-worming efficacy, seem to be functioning here as guilt by association. I can't see any reason they should be extrapolated to all other calculations in different chapters of a >1000 page document. The figure from DCP2 for trachoma treatment directly references the primary source, so it's highly unlikely to be vulnerable to any spreadsheet errors.
The table Toby cites and you reference here (Table 50.1 from DCP2) says "trichiasis surgery". This means surgical treatment for a late s... (read more)