4

Drug liberalization ballot initiative cost-effective at $460/DALY

Cross-posted to the Enthea site.

 

In a nutshell

  • A California ballot initiative that increases access to psychedelic drugs is a promising intervention in terms of DALYs averted.
  • We estimate the cost-per-DALY-averted to range from $26/DALY to $90,515/DALY, with a best-guess estimate of $460/DALY. Full analysis available here.
  • $460/DALY is our best-guess cost-effectiveness for the medical benefits of a California drug liberalization initiative.
    • Many benefits are not included in the analysis (e.g. pushing US federal drug policy away from a crime model & towards a public health model), as they are speculative and difficult to model. We believe the speculative benefits may actually outweigh the medical benefits of an initiative, but that question is not considered in our model.

 

Q & A

What intervention is being modeled, exactly?

  • The analysis models the costs & benefits of a ballot initiative that increases access to psychedelics such that sufferers of depression, PTSD, alcoholism, and tobacco addiction can receive psychedelic treatment for these conditions in California.
  • The analysis is intentionally agnostic re: the policy specifics of such an initiative. We are still learning about the strategic landscape here, and are currently unsure what specific initiative text would be best.

Why model mental health impacts if the largest benefits would come from other impacts?

  • The mental health benefits from psychedelics are one of the least speculative benefits of liberalizing US drug policy. Our model gives the expected benefit a ballot initiative would achieve if all speculative impacts were net zero.
  • Simpler to model: other impacts are complicated by considerations like flow-through effects. It would be very difficult to build a believable quantitative analysis of these impacts.

Is a drug liberalization ballot initiative cost-effective on the grounds of mental health benefits alone?

  • GiveWell top charities achieve cost-per-DALYs on the order of $100s per DALY. However, these results rest on philosophical assumptions about population ethics (for AMF, discussion here (a) and here (a)) and empirical interpretations of the interventions' effects on happiness and income (for GiveDirectly and deworming interventions, some discussion here (a)).
    • We don't take a position on these issues here; we raise them only to point out that arriving at apples-to-apples comparisons of heterogenous interventions is very complicated, and that there are reasonable assumptions under which GiveWell top charities are not extraordinarily cost-effective compared to other interventions.
    • Mental health interventions have the benefit of directly increasing life satisfaction (which physical health and economic development interventions can only do indirectly).
  • Our best-guess is that the medical benefits that flow from a drug liberalization ballot initiative have a cost-effectiveness of $460/DALY. This is competitive with the cost-effectiveness of GiveWell top charities.

Why doesn't the model account for adverse effects of psychedelics, like bad trips?

  • Contrary to common belief, psychedelics are quite safe, and very nontoxic – see the “Background and safety” section here (a), as well as the “Safety of psychedelics” section of Nichols 2016.
  • Carbonaro et al. 2016 surveyed 2,000 psychedelic users who self-reported having a “challenging experience” with psychedelics. Despite the difficulty of the experience, 84% reported benefiting from the bad trip. Given this result, it's not obvious that bad trips are pathological.

One plausible strategy for increasing access to psychedelics is to decriminalize all drugs. Wouldn't all-drug decriminalization cause a major uptick in use of addictive drugs?

  • It's definitely possible, though Portugal decriminalized all drugs in 2001 and didn't see an uptick in drug use.
  • Assessing this impact is complicated – all-drug decriminalization would increase access both to addictive drugs (e.g. opioids) and psychedelic anti-addiction treatments; it's unclear what the net impact of this would be.

 

Release notes

  • The model assumes that a ballot initiative polls well. Based on private polling we have seen, drug liberalization policy proposals are very popular in most US states.
  • Compared to our minimum-benefit model, our best-guess model yields a much better cost-effectiveness ($460/DALY compared to $119,000/DALY).
  • This change is largely driven by no longer including costs of treatment in the model. In our minimum-benefit model, we included costs of treatment to avoid issues around charitable leverage (a).
  • However, we no longer think it's appropriate to include all costs of treatment in the model.
    • Unlike global health interventions, where nearly all costs are covered by philanthropic actors, much of the treatment costs of psychedelic-assisted psychotherapy in the US will likely be covered by patients & insurance companies. These actors are not acting philanthropically, so it makes less sense to consider their contributions in the context of donor coordination.
    • Our best-guess model includes an adjustment – "percent of impact attributable to campaign." This adjustment is necessary as it's not accurate to attribute 100% of the benefit of psychedelic-assisted psychotherapy to the campaign that legalized the intervention.

Comments (23)

Comment author: ea247 15 January 2018 01:33:10AM 4 points [-]

I echo Michael Plant's sentiments. I'm glad you're quantifying the benefits of this potential intervention.

I started looking through the CEA and thought it seemed optimistic in various ways, but then I realized I could just look at the end number and see if, even without adjustments, it beat GiveWell recommended charities. Unfortunately it doesn't. You said that GiveWell's charities are in the range of hundreds of dollars per DALY and that didn't gel with my memory. I looked it up and AMF is around $1,965 per life saved, equivalent to 36 DALYs, so 1,965/36 = ~$54/DALY. SCI was $1,080 per life saved, so $1,080/36 = $30/DALY. GiveDirectly is $11,663/36=~$323/DALY, but the reason they recommend GiveDirectly is in part because it has a lower barrier to prove itself in evidence because it is a very "direct" intervention. (source: https://docs.google.com/spreadsheets/d/13b_qt-G_TQtoYNznNak3_5dzvzgCSUPJnk3l5dMisJo/edit#gid=1034883018) These numbers hold up roughly when you look at estimates from 2012 when they were still using DALYs.

Nevertheless, if you are considering where to donate, your best guess estimate is less cost-effective than GiveWell interventions. This is before it goes through the rigors of a GiveWell CEA as well, which would definitely have less optimistic numbers, especially given the low evidence base.

I'd like to end on a note that I think that posting new cause areas to the EA movement is scary because it's a critical minded bunch, so hats off to you for having the courage to do so. Keep trying; I commend you for it. Unfortunately, even if you defend it as well as it can be defended, it might not win compared to the existing top charities. However, if nobody does this work, no new causes will be "discovered", and so even if it doesn't win, this sort of work is very likely to be net good in expectation.

Comment author: Milan_Griffes 15 January 2018 03:40:21AM *  2 points [-]

Thanks for the thoughts.

I started looking through the CEA and thought it seemed optimistic in various ways

It would be helpful if you could point out places where our best-guess value seems optimistic. The model does include a pretty steep evidence discount – best-guess assumes just a 20% chance that each effect replicates.

I looked it up and AMF is around $1,965 per life saved, equivalent to 36 DALYs, so 1,965/36 = ~$54/DALY

As mentioned in our post, GiveWell has moved away from the DALY framework, so it's not clear that a simple conversion like this is the way to convert its model outputs into DALYs. (We've asked GiveWell for clarification on this.)

the rigors of a GiveWell CEA as well, which would definitely have less optimistic numbers, especially given the low evidence base.

Why do you think a GiveWell CEA would definitely yield less optimistic numbers?

... it might not win compared to the existing top charities.

I don't think this is a great way to think about comparing charities. Quantitative models are complicated & very sensitive to their input parameters, so a "winning" charity may only be winning because of the way a model is structured.

This isn't to say that quantitative comparisons aren't useful. Instead, I think quantitative comparisons are useful for winnowing out interventions whose cost-effect falls orders of magnitude below that of top charities. But I don't think the fidelity of any quantitative model we use today is sufficient to discern the best intervention between interventions on the same order of magnitude.

It becomes even trickier to think about when comparing interventions across very different domains. For example, x-risk interventions either dominate global health interventions (if you take cost-effectiveness estimates literally & are a total utilitarian), or aren't competitive at all (if you only believe cost-effectiveness estimates above some threshold of rigor, so aren't compelled by back-of-the-envelope estimates that massively favor x-risk).

In practice, it seems like the EA community gets by without making direct effectiveness comparisons between x-risk & global health interventions, and instead houses both as priority cause areas.

Something like this is my hope for drug policy reform – a sufficiently compelling case is articulated such that EA decides to house it as a priority cause area (already done to some extent, see Open Phil's grants to the Drug Policy Alliance: https://www.openphilanthropy.org/focus/us-policy/criminal-justice-reform/drug-policy-alliance-drug-decriminalization). It doesn't seem necessary to "win" the cost-effectiveness comparisons, only demonstrate that its cost-effect is competitive under plausible assumptions.

Comment author: ea247 16 January 2018 06:38:00PM 2 points [-]

The reason why interventions like AR or x-risk are accepted by the EA movement (although not by all EAs) is that from a CEA perspective they do better than GiveWell top charities. The reason a lot of people still don't accept them as interventions though is because people discount based on evidence base differently, with some people taking non-evidence based CEAs more seriously than others. If drug policy does worse from a CEA perspective than GiveWell, AR and x-risk, and is worse from an evidence perspective than GiveWell charities, where is its advantage?

You could make a case that it's better from a metric perspective (ie preventing unhappiness through depression rather than DALYs which has issues with it, like over-valuing preventing death according to a lot of value systems), but deworming improves lives; it doesn't prevent death. Same with GiveDirectly.

For giving detailed feedback on the CEA, I unfortunately just don't have the energy to do the full thing, but if the final number still isn't enough to make me switch from GiveWell charities, it doesn't make sense to look more into the details. However, one thing that jumped out to me that others mentioned was the chance of the ballot coming through. I think looking up the historical rate of ballot initiatives being passed would be a good thing to look into.

Comment author: MichaelPlant 16 January 2018 11:14:25PM 2 points [-]

On your original comparison to GW charities, I wouldn't just take GW's analysis as the canonic truth on the matter. Their CEA is pretty complicated, but ultimately they value charities based on how well they either 1. save lives or 2. increase consumption.

What you think about the value of saving lives is a philosophical question. I've written about this elsewhere so won't repeat myself. What you think about the value of increase consumption (SCI and GD) is probably an empirical question. If you value happiness then increasing consumption is a really bad way to increase aggregate happiness because of adaption and comparison effects (I discuss this in my EAG talk).

When I think about GW charities, I'm am highly sceptical they do much good at all. I know, highly controversial....

I say this because it opens the space to look at other things, like mental health and pain, both of which drug policy reform help.

However, even if you take GW's calcs at face value (I'm not sure exactly what that is) I think it would still be possible to build an EV calc for drug policy reform that rivals them. In this post I suggest a campaign for rescheduling psychedelics could spend £250 billion and be competitive with AMF. Milan's calcs are really helpful because it's important to start filling in the details of this analysis. In many ways, Milan's is more complete than mine, which is quite simple.

Comment author: ea247 17 January 2018 12:53:00AM *  2 points [-]

Fair point, that deworming and cash transfers increase consumption instead of directly increase well being, or at least that's what GiveWell's main analysis rests on. I do recall that the GD study actually did look at SWB and on page 4 (bit.ly/2B97A1Y) it says that it increased a bunch of different happiness metrics as well (depression, stress, happiness and life satisfaction). However, if you only looked at that effect, GiveDirectly may not be that cost-effective. I haven't investigated it that much from that angle.

In terms of preventing infant mortality, it seems unlikely that losing a child wouldn't cause immense suffering to the parents, especially the mother. People often think that this wouldn't happen because people just "get used to" babies dying, but the odds that a child will die is actually quite low nowadays, even in the developing world. In India, where I have the most experience, it's measured in deaths per 1,000 live births, not 100, because it's that's rare. Additionally, because I don't think death is nearly as bad as DALYs would have it, I looked a lot into parental mourning before choosing SMS reminders. I don't have anything formal I wrote up I can point to (though I might at some point), but my research found that most parents, after the loss of a child, are depressed for around a year, with some tail ends of people who never appear to recover.

If it's the metrics issue that's leading to drug policy reform, I would recommend looking into preventing iron deficiency (through supplements or fortification) as an alternative. It's more evidence based and iron deficiency causes massive unhappiness. Anecdotally I've had friends who transformed from sad grumpy monsters into happy productive members of society after realizing they were deficient. Additionally there's evidence it increases income, increases IQ if taken during pregnancy, and decreases mortality in certain circumstances, so it's pretty robust no matter the metrics you care about.

Lastly, I'll admit that I haven't read all of your posts / critiques of AMF's effectiveness, so I'll have to go and do that :)

Comment author: Milan_Griffes 17 January 2018 01:52:20AM *  0 points [-]

If drug policy does worse from a CEA perspective than GiveWell, AR and x-risk, and is worse from an evidence perspective than GiveWell charities, where is its advantage?

My view here relies on some background that isn't fully articulated yet. (I'm in the process of doing so with this series on consequentialist cluelessness: 1, 2, 3)

The basic thrust is that psychedelic experiences can function as a catalytic engine for social change, both by improving the motivations of highly capable but insufficiently reflective people, and by improving the capabilities of well-intentioned people who struggle with internal blockers.

Given cluelessness, this approach seems promising because increasing the number of highly motivated, well-intentioned people has more error correction baked in than interventions where the case for their effectiveness is driven by their proximate impacts.

Comment author: MichaelPlant 17 January 2018 12:40:35PM 4 points [-]

The basic thrust is that psychedelic experiences can function as a catalytic engine for social change, both by improving the motivations of highly capable but insufficiently reflective people, and by improving the capabilities of well-intentioned people who struggle with internal blockers.

If that's what you think does the real work of drug liberalisation, you should probably state that and build a cost-effectiveness model on that basis, rather than try to justify drug reform on other terms but with that as the true motivation. I, for one, am pretty sceptical, because I can't imagine loads more people would, counterfactually, start taking drugs recreationally and that, for those that do, this will have much impact on their cognitive powers.

Comment author: Milan_Griffes 17 January 2018 03:33:54PM *  0 points [-]

I, for one, am pretty sceptical, because I can't imagine loads more people would, counterfactually, start taking drugs recreationally and that, for those that do, this will have much impact on their cognitive powers.

There have been many historical examples of norms around drug use in the US changing quickly (e.g. 19th century teetotalism, 1960s psychedelia). What makes you skeptical about another change in norms?

If by "cognitive powers" you mean adding IQ points, I agree that psychedelic experiences won't help much with that. Psychedelics can cause pretty powerful shifts in a person's motivations & worldview (this is strongly supported by anecdote; see also Griffiths et al. 2008). I think poor motivations and problematic worldviews are more often blockers than someone's IQ.

Comment author: MichaelPlant 17 January 2018 04:14:32PM 1 point [-]

Hmm. Yes, I agree cognitive shifts could be pretty powerful from psychedelics and that IQ points probably won't change. I think I misread you.

The larger part of my scepticism is my intuitive hunch that loads of people wont suddenly start taking psychedelics if they're legal/decrimed. This isn't a strongly informed judgement and I could probably change my mind if presented with compelling reasons.

On the worldview stuff, if the idea is something like "people take drugs and this changes how they think for the better", which I actually think is pretty plausible, a particular challenge is that those who you, I expect, would most like to take such drugs, i.e. the very close-minded, are probably going to be the least likely to take them anyway.

Comment author: Milan_Griffes 22 January 2018 10:33:04PM *  0 points [-]

loads of people wont suddenly start taking psychedelics if they're legal/decrimed

I agree that liberalized drug policy is not sufficient to increase the number of people having psychedelic experiences, but it's a prerequisite of many promising interventions in this area (e.g. setting up US-based psychedelic retreat centers).

Comment author: Lila 18 January 2018 04:33:00AM 0 points [-]

I'm not sure how the beliefs in Table 3 would lead to positive social change. Mostly just seems like an increase in some vague theism, along with acceptance/complacency/indifference/nihilism. The former is epistemically shaky, and the latter doesn't seem like an engine for social change.

Comment author: Milan_Griffes 18 January 2018 06:34:24PM 0 points [-]

I think the more interesting results are in Figure 2, which show increases in life-satisfaction & positive behavior change that endured 14 months after the intervention.

Comment author: Milan_Griffes 17 January 2018 03:16:19PM *  0 points [-]

If that's what you think does the real work of drug liberalisation...

I think both the proximate mental health benefits & the speculative catalytic engine benefit do work. The case for the mental health benefits has more empirical basis & is easier to model, so I started with that.

Comment author: Milan_Griffes 22 January 2018 10:30:40PM 0 points [-]

Update on how to convert GiveWell's model outputs to DALYs: we asked someone familiar with GiveWell's 2018 cost-effectiveness model about this.

They weren't comfortable being quoted; the gist of their reply is they can't think of a straightforward way to convert from GW model outputs to DALYs that they'd be comfortable using formally.

Comment author: MichaelPlant 14 January 2018 01:20:42PM *  2 points [-]

Thanks for doing this. I found the way you'd quantified the treatment benefits very useful.

Two sets of comments:

1

I would have found it very helpful if you'd explained the context of the intervention. Some questions that sprung to mind:

what are ballot initiatives in Calfornia and how do they work? is this to raise funds to put this up for a vote? If so, what are the costs involved and why do they vary? Is this to put this to put it up for a vote and campaign for it? Something else?

What exactly do you mean by 'drug liberalisation'? Changing the medical laws, the recreational ones? If so, in what way? I understand you want to be 'agnostic' on the details, but I'd say you're closer to being 'vague' as I'm really not sure what you have in mind. Another way of being agnostic would be to say "there are options A to F of how this could work in practice, we don't know which is best and it would/wouldn't change the model for these reasons"

2

I'd also have found it helpful if you'd explained what you're doing at the various steps of the model and why. I assume that anyone, such as myself, interested to look through the model would also read a text explanation and this would aid comprehension. Important counterfactuals to consider seem to be:

-How many years of benefit there are before this happens anyway. Your answers were 2, 5 and 10 years. Could you explain your thinking there and what it's sensitive to?

-Why did you model the effect just in California? I think most of the benefit of something like this is that it would speed up drug reform around the world, i.e. what California does today, the world does tomorrow. It would be good to have something, even vague, on how much better this looks if we include a domino effect.

-You state this would need $6m, $14m or $38m in funding. In addition to knowing exactly what that is funding for, I'd want to know about the expected value at different sums raised. i.e. if I can just scrap together $100k, am I right in thinking this is a waste of time? If that's true, that changes who the relevant audience is and it would only be major funders.

-You mention depression but not, as far as I can see, anxiety. Is there a reason for that?

-Are you assuming the people who would be treated with psychedelics wouldn't otherwise receive treatment? What are you assuming here?

I might have other things, but that's probably enough for now.

Comment author: Milan_Griffes 14 January 2018 07:59:33PM *  2 points [-]

Replying to the second set:

years of benefit there are before this happens anyway. Your answers were 2, 5 and 10 years. Could you explain your thinking there and what it's sensitive to?

"Before this happens anyway" is complicated to think about & we don't have a lot of information about how things will unfold over the next decade. That said, two considerations here:

  • What portion of the benefit flows from FDA approval of psychedelic-assisted psychotherapy (in process, championed by MAPS for MDMA & Usona for psilocybin)?
  • How likely is it that another drug liberalization initiative occurs counterfactually?

Currently, we don't have good answers to these questions (we are continuing to think about them). Our best-guess is that a liberalization initiative yields roughly 5 years of counterfactual benefit, and the "optimistic" and "pessimistic" scenarios are arbitrary adjustments around our best-guess.

I think most of the benefit of something like this is that it would speed up drug reform around the world, i.e. what California does today, the world does tomorrow.

Agreed, though we haven't figured out a way to believably model knock-on effects like this, so have left them out to be conservative. (Suggestions welcome!)

I'd want to know about the expected value at different sums raised.

We haven't put together a funding levels analysis yet, though plan on doing so. Polling & focus group research is pretty useful, independent of the signature collection and media campaign costs.

i.e. if I can just scrap together $100k, am I right in thinking this is a waste of time?

$100k would cover a lot of polling & qualitative research; I don't think that would be a waste of time. Funding at that level could also cover core operating costs for several months.

One comparison point here is the Brexit "Leave" campaign, which was run very cheaply compared to the "Remain" campaign but still clocked in at £13.5 million (see p. 5 of https://dominiccummings.files.wordpress.com/2017/01/20170130-referendum-22-numbers.pdf )

You mention depression but not, as far as I can see, anxiety. Is there a reason for that?

Fewer high-quality studies on anxiety reduction, if I recall correctly.

Are you assuming the people who would be treated with psychedelics wouldn't otherwise receive treatment? What are you assuming here?

This is a good question. For smoking cessation & alcoholism, the model only looks at people who failed to quit by other means. For depression & PTSD, it considers all treatment-seekers. We should add an adjustment to account for the effect of counterfactual treatment. Thanks!

Comment author: Milan_Griffes 14 January 2018 07:45:42PM *  1 point [-]

Thanks for the thoughtful comments :-)

Replying to the first set:

very helpful if you'd explained the context of the intervention.

Our approach so far has been to publish research products as we finish them, rather than holding off until we have a fully articulated case for the intervention. A few reasons for this:

  • "Publish as we go" allows more opportunities for people to give input on our work.
  • I think fewer people would engage with the research if we released it as one, very long post.
  • We've found it more motivating to publish smaller products as we complete them.

what are ballot initiatives in Calfornia and how do they work?

There's a bunch of material on this elsewhere, we're not prioritizing writing up another explainer at this time. For example: https://ballotpedia.org/California_2018_ballot_propositions

What exactly do you mean by 'drug liberalisation'? Changing the medical laws, the recreational ones?

Probably decriminalization, but we're still exploring here. We're using "liberalization" as a blanket term to refer to policy changes that increase access to drugs.

Comment author: jimrandomh 13 January 2018 11:01:35PM 2 points [-]

In this model, what is the probability that the initiative (which I see is modeled as costing $6-39M) is successful? Or is it assumed that in the case where it isn't going to succeed, the cost is limited to the cost of polling ($50-300k)?

Comment author: Milan_Griffes 13 January 2018 11:46:20PM 2 points [-]

The probability that the initiative succeeds is given on row 73. Our best-guess is 80% chance of success, conditional on raising enough funding & future polling remaining good.

Our best-guess here is informed by private polling we have seen that suggests a drug liberalization initiative would be very popular in most US states.

Comment author: Carl_Shulman 14 January 2018 06:46:34PM 2 points [-]

Did you collect base rate information for other initiatives before campaigns (which tend to lower approval relative to pre-campaign polling) for that parameter?

Comment author: Milan_Griffes 14 January 2018 07:36:51PM *  0 points [-]

We haven't looked at base rates yet, mostly because it's a big project & we're capacity constrained.

Two complications around selecting the right base rate:

  • Unclear what set of campaigns to consider (all pro-liberalization ballot initiatives in the US? only those in California? only those in a particular route, e.g. decriminalization or legalization?)

  • Even if we considered the broadest set of campaigns, the n is going to be small. (probably less than 30?)

That's not to say looking at the base rate isn't worth doing, only that other projects have been prioritized above it.

Comment author: Lila 15 January 2018 05:29:14PM 2 points [-]

Modeling the risk of psychedelics as nonexistent seems like a very selective reading of Carbonaro 2016:

"Eleven percent put self or others at risk of physical harm; factors increasing the likelihood of risk included estimated dose, duration and difficulty of the experience, and absence of physical comfort and social support. Of the respondents, 2.6% behaved in a physically aggressive or violent manner and 2.7% received medical help. Of those whose experience occurred >1 year before, 7.6% sought treatment for enduring psychological symptoms. Three cases appeared associated with onset of enduring psychotic symptoms and three cases with attempted suicide."

Comment author: Milan_Griffes 15 January 2018 05:42:13PM *  1 point [-]

First thing to note is that Carbonaro et al. 2016 surveyed recreational psychedelic users who were using psychedelics in uncontrolled settings, whereas the intervention we're modeling is psychedelic-assisted therapy where treatment is administered under the supervision of trained attendants.


Carbonaro et al. 2016 was an online survey of psychedelic users who self-reported a bad trip, so there's definitely sampling bias here (though unclear in what direction... users who had really bad trips could be more excited about debriefing their experience; user who had really bad trips may also have been unable or unwilling to take the survey).

It's unclear how 11% could put self or others at risk of physical harm, with only 2.6% behaving aggressively and 0.15% attempting suicide.

2.6% behaving aggressively is 52 people out of 1993. It's probably worth adding an adjustment to the model for this, thanks for pointing it out.

Three cases out of 1993 survey takers is a 0.15% rate of psychosis onset, and a 0.15% rate of attempted suicide, which is probably low enough to not be meaningfully different than base rates (but again, sampling bias here).