Comment author: MichaelPlant 04 December 2017 12:37:17AM *  2 points [-]

Hello Elizabeth, thanks for writing this up. I think this is important work so please take all my points belows as friendly suggestions for improving the methodology so we can get a better answer or just clarificatory questions because I don't know what something is (I always find it quite hard to understand other people's CEA models).

Saying MBSR will have an effect for 38 years after treatment seems extremely generous. Do you have any data either way on how long the benefits of mindfulness last for? I've seen stuff saying CBT works for 5 years on depression(/anxiety) without much of a drop, but 38 years is very long.

What is 'time cost of initial work'?

What does 'negative years of life after treatment' refer to?

The effects of MBSR on deprssion/anxiety (or is it just anxiety? I haven't check the studies yet) you report are much weaker than I expected. The 7% number suggests that just 7% of the 'DALY-badness' of anxiety has been removed, suggesting it makes a dent in, rather than 'cures', the condition. Do you know what's going on here? Is MBSR partially effective (and how does this compare to CBT)? Are these perhaps studies on low anxiety people that got completed cured? Something else? I would think the effect, while it lasts, would have a much higher impact.

Time cost for continued practice seems odd to me. First, it's pretty implausible every person who went on an MBSR course would do 1 hour's practice each day (and very implausible if you assume they will do this for a next 39 years!). Second, you seem to be assuming the DALY weight of each hour meditating is 0.5, which is roughly as bad as it is to be anxious anyway, no? Surely time meditating can't be that painful. Unless you think meditation is actually unpleasant, something people suffer through to get less stressed when not meditating, I'd remove that part of the CEA. Meditation seems neutral/good IME. The appropriate counterfactual is how pleasant is meditation vs whatever they would have been doing instead with that time (e.g. watching tv?).

Your model also seems to assume that, if not for the treatment, the person wouldn't have had MBSR at all. Given the spread of mindfulness practice worldwide, I think this is better thought of as "if we fund this intervention, how much earlier will it cause 1 average person to start practising mindfulness than they otherwise would have?" If they person would have been an avid mindfulness-er 5 years later anyway, the effect is just 5 years. There's also the possible counterfactual that teaching this one person caused them to speed up the spread of mindfulness because they pass it on to their friends. And there's the possibility they would have used something else, such as CBT, to treat their depression/anxiety, anyway rather than left it untreated. Or that their depression/anxiety would have ended naturally. I'm unsure how to work through these counterfactuals, but it ought to be flagged even if you ultimately say "I'm just going to leave aside these counterfactual effects as too complicated".

Another worry comes when I ask myself "would this be a good thing for EAs to fund?" It seems anyone with access to the internet could self-teach mindfulness if they really wanted to. Hence the relevant obstacles are that people don't want to do it or aren't aware of it. I doubt there are hordes of people who know about MBSR and would do it but are lacking the funds to pay for the course themself. In the developed world, people could probably cough up $300 themselves. It seems a bit weird for EAs to be paying for the medical treatments of other people in the developed world. Suppose, instead, this is a medical treatment to be offered the depressed/anxious in the developing world. Then my concern is one of cultural barriers and that take up of mindfulness would be quite low (intuitively, this seems like a bigger problem for mindfulness than CBT).

If the true obstacles aren't money but awareness or motivation, that suggests the better things for EAs to do might be paying for public campaigns that advertise mindfulness, e.g. via Developed Media International. My concern is then neglectedness: there are(/will be) companies trying to market mindfulness to people for a profit. If this is true, EAs might want to leave this to the market to provide and do something else. I'm not quite sure how to think about this either.

Comment author: Elizabeth 04 December 2017 07:09:05AM 2 points [-]

These are all great comments, thank you.

Saying MBSR will have an effect for 38 years after treatment seems extremely generous. Do you have any data either way on how long the benefits of mindfulness last for? I've seen stuff saying CBT works for 5 years on depression(/anxiety) without much of a drop, but 38 years is very long.

The only studies I remember (and it's been a while since I did this) checked in at 18 months and still showed significant improvement. I'd be very interested if you have a source for the CBT numbers, since that seems directly applicable.

What is 'time cost of initial work'?

Time to attend the class.

What does 'negative years of life after treatment' refer to?

guesstimate doesn't have a sigmoid function, so I used a constant minus a lognormal range.

The effects of MBSR on deprssion/anxiety (or is it just anxiety? I haven't check the studies yet) you report are much weaker than I expected.

All of the studies I read were on mild-to-moderate depression or anxiety, and ended with people having no-to-mild depression/anxiety on average. It wouldn't surprise me if in reality some people got very large benefits and others got none, which would make this a somewhat better intervention because people who didn't benefit could stop.

you seem to be assuming the DALY weight of each hour meditating is 0.5, which is roughly as bad as it is to be anxious anyway, no? Surely time meditating can't be that painful.

I think treating it as bad as death is a ceiling we won't reach, but I do think many people find meditation actively unpleasant, especially if they have a mood disorder, relative to doing something distracting. If meditation were more fun than watching TV we wouldn't have to justify it with long term health.

Another worry comes when I ask myself "would this be a good thing for EAs to fund?" It seems anyone with access to the internet could self-teach mindfulness if they really wanted to.

The costs of learning are so low relative to the costs of practicing they end up not making a big difference in the model.

It seems a bit weird for EAs to be paying for the medical treatments of other people in the developed world.

I am surprised to hear you say that, given your emphasis on mental health, which is suffering independent of material circumstances. Given that only 40% of Americans have the savings for an unexpected $1000 bill (https://www.prnewswire.com/news-releases/63-of-americans-cant-afford-500-car-repair-or-1000-emergency-room-visit-300200097.html) , I find it quite realistic that people would benefit from others paying for care, although I expect the time to take the class and practice to be even bigger barriers.

Then my concern is one of cultural barriers and that take up of mindfulness would be quite low (intuitively, this seems like a bigger problem for mindfulness than CBT).

My intuition is the opposite: CBT was developed in a very specific cultural context and seems to take as an assumption that nothing is actually wrong. Mindfulness meditation has been practiced for... long enough that getting an actual date is infeasible.

If the true obstacles aren't money but awareness or motivation, that suggests the better things for EAs to do might be paying for public campaigns that advertise mindfulness, e.g. via Developed Media International. My concern is then neglectedness: there are(/will be) companies trying to market mindfulness to people for a profit. If this is true, EAs might want to leave this to the market to provide and do something else. I'm not quite sure how to think about this either.

My research on media interventions was very pessimistic, even for DMI (http://effective-altruism.com/ea/1ha/mental_health_shallow_review/). This time could be different, but I think it's a very hard problem.

I think you raise a valid point that there's a vibrant market in mindfulness apps, and I'd be very curious to know how they stacked up to the formal program.

Comment author: Alex_Barry 03 December 2017 04:29:31PM 2 points [-]

Thanks for the post, as a minor nitpick, shouldn't the maximal DALY cost of doing something for an hour a day be 1/16, since there are only 16 waking hours in a day and presumably the period whilst asleep does not contribute?

Comment author: Elizabeth 03 December 2017 04:53:14PM 0 points [-]

You're the second person to argue for this (other was on my personal blog), and I hear the argument. I think there's a slippery slope of what to control for here- if I include sleep, I'd also want to look at how happy people were when meditating relative to the activity it displaced.

Comment author: DonyChristie 27 November 2017 09:48:25AM 0 points [-]

Did you look into coherence therapy or other modalities that use memory reconsolidation? It is theoretically more potent than CBT.

Comment author: Elizabeth 27 November 2017 10:46:17PM 1 point [-]

How scalable is coherence therapy? CBT is attractive not just because of the experimental backing but because it can be automated.

Comment author: Ben_Todd 22 November 2017 10:53:11PM 3 points [-]

Hey, on (1), cost-effectiveness estimates and wellbeing estimates are useful, but I'd also want to think more broadly about the INT framework side-by-side with global health, as well as other more qualitative arguments on each side.

Relatedly, I can imagine concluding that it's a big and neglected problem, but also one where clear, evidence-backed scalable interventions don't yet exist, so the top priority might be more research to develop better interventions. This perspective might get overlooked if you focus more on cost-effectiveness estimates, but seems pretty reasonable in some of these areas, such as mindfulness.

If that perspective is correct, then the question becomes is it better to scale up proven global health interventions, or do more research into mental health interventions?

Comment author: Elizabeth 22 November 2017 11:02:47PM 1 point [-]

...is it better to scale up proven global health interventions, or do more research into mental health interventions?

That's a great question I don't know enough to answer. I'd love to see general guidelines for deciding on investment in research versus boots on the ground work.

Comment author: Ben_Todd 22 November 2017 06:06:57AM 6 points [-]

Great post, like all the data, and would be keen to see more work like this. I've added a link to it here: https://80000hours.org/articles/cause-selection/

Some more questions I'd be interested in:

1) I'd be interested to see more on how you think it compares to other EA causes all considered, especially the most similar one, global health. I'd start with taking a short-term DALYs and economic perspective, but would also be interested in what a long-term perspective might say about the comparison (I've almost seen nothing about this)

2) I'd be interested to see firmer recommendations for people who have already decided they want to focus on mental health - what are your thoughts on the most promising interventions and career paths?

Comment author: Elizabeth 22 November 2017 04:17:56PM *  3 points [-]

Thanks Ben, this is very helpful as I revise the report.

For (1): my research showed that as bad as the existing options for measuring utils are, they're worse for mental health. I'd originally published this on my own blog to avoid spamming EA Forums, but this is the second time I'm linking to it in the comments so I've posted it hear as well: http://effective-altruism.com/ea/1he/measuring_the_impact_of_mental_illness_on_quality/. You might also be interested in a DALY/$ estimate I made for Mindfulness Based Stress Reduction: https://acesounderglass.com/2017/11/20/cost-effectiveness-of-mindfulness-based-stress-reduction/ .

For (2): I'll update the doc to reflect this, but in order of increasing abstraction:

'1. Copy StrongMind's model to other poor countries. A SM representative I talked to said they would be happy to help get other charities off the ground.

'2. Research how to optimize mindfulness, make it palatable to the general public, and expand the circle of people it helps/has been demonstrated to help. I met an organization working on this for business at a conference and am quite excited at the potential for increasing both happiness and productive capacity, although I know nothing about the implementation.

'3. Figure out how to measure subjective experience so we can better compare interventions, within and without mental health.

Comment author: Peter_Hurford  (EA Profile) 21 November 2017 09:57:50PM 0 points [-]

Do you know why StrongMinds chose the intervention that they did? Do you think there's a cost-effective way to administer CBT in the third-world?

Comment author: Elizabeth 21 November 2017 10:32:06PM 2 points [-]

According to the person I talked to, StrongMind's chose their intervention because it was the only one that had experimental backing equivalent to what J-PAL does.

Unorganized thoughts on CBT for the bottom billion: *Poor Americans complain that it implicitly assumes the problem is in your head and is unhelpful if there's an actual problem in your life. I'd guess this would be worse for the bottom billion. *I'd expect significant changes to have to be made to adjust to local cultures. *The maximum cost would be in the ballpark of StrongMinds. The minimum cost would be much lower, I expect CBT to be more amenable to scaling via phones. *I am pretty skeptical of automated CBT. I think it follows the letter of CBT but is missing some implicit step that is actually what is helpful. This is mostly an intuition. *OTOH, it could have a very low success rate and still be very helpful if enough people tried it and the cost of trying was low. *CBT has been getting less effective as time goes on, potentially because the best insights have made their way into popular culture. I wouldn't expect that to be true for the bottom billion, which would increase its effectiveness.

So I think my ultimate answer is "CBT is a good jumping off point to look for something that would indeed be very cost-effective in the third world."

Comment author: Milan_Griffes 21 November 2017 01:56:29AM 1 point [-]

Great post; very excited to see more good work in this direction.

Milan Griffes's post on psychedelic legalization on the same forum, in which he estimates the return to lobbying to legalize psychedelics at $52,000-$442,000/DALY.

Probably worth noting that $52,000-$442,000/DALY is the all-in cost which includes costs of treatment, in addition to the cost of a ballot initiative. Treatment costs make up almost the entirety of the total cost, and it's unclear who would bear these costs (funders of the ballot initiative almost certainly wouldn't).

Comment author: Elizabeth 21 November 2017 02:51:00AM 1 point [-]

Oops, thanks for the correction. Do you have those broken out separately?

Comment author: pasha 20 November 2017 08:43:45PM 2 points [-]

I like this report and it's a good start. In some ways i wonder if even this report underestimates the real costs. Mass shootings, divorce, funding of crime, poor financial choices are things that come to mind as negative flow-through effects of poor mental health.

With that in mind, I find that it is difficult to even talk about mental health interventions without a good metric. Reducing number of suicides is great, but it's not going to capture large groups of people going through "slightly depressed" to "not depressed," unless the intervention is correlated with reducing suicides as well. Things like improving self-reported happiness are probably good, but it's not clear that's captures what we mean by mental health.

I feel that the next step in improving the situation is more along the lines of figuring out metrics that we can measure that specifically target mental health.

Similarly for interventions, there is probably a large number that are hard to evaluate at this point. Interventions are generally in two categories: biological and cultural with biological being the easy one. Things like diet, promoting better sleep / less screen time. Cultural would be more tracking how communities evolve and help / hurt mental health and what can be done to promote them.

Comment author: Elizabeth 20 November 2017 09:28:03PM 4 points [-]

Double checking that you saw the supporting doc on the difficulties in measuring mental health at https://acesounderglass.com/2017/11/20/measuring-the-impact-of-mental-illness-on-quality-of-life/

Comment author: Elizabeth 24 June 2017 05:23:01PM 0 points [-]

I think costly signaling is the wrong phrase here. Costly signaling is about gain for the signaler. This seems better modeled as people trying to indirectly purchase the good "rich people donate lots to charity.". Similar to people who are unwilling to donate to the government (so they don't think the government is better at spending money than they are) but do advocate for higher taxes (meaning they think the government is better at spending money than other people are). They're trying to purchase the good "higher taxes for everyone".

Comment author: Peter_Hurford  (EA Profile) 27 April 2017 02:03:01AM 2 points [-]

It's worth noting that it's all pretty fungible anyway. GiveWell could have just as easily claimed the money was going toward an incubation grant and then put more incubation grant money toward AMF.

Comment author: Elizabeth 27 April 2017 02:48:52AM 5 points [-]

This seems like an excellent reason to have someone uninvolved with an existing large organization administer the fund.

View more: Next