Worldview uncertainty

In this post, I will apply two methods used in dealing with moral uncertainty to a different problem, which is the one of choosing between radically divergent worldviews.

Imagine a young and altruistically motivated graduate. As she learns more, her circle of moral concern expands from people near her to all humans, and then to animals. At some point, she encounters arguments for prioritising the long-term future. Suddenly, all her previous efforts seem futile and it takes her mere seconds to decide that she wants to put her life in service of creating a good future. Just as she embarks on pursuing her completely re-ordered priorities, she halts for a second to contemplate the train of thought that led her thus far. She knows that not many people share her new priorities, and that most have radically different worldviews from hers. How certain can she be that she has not made a mistake on the way?

The above is an example of the kind of question discussed in the 80.000 hours podcast with Holden Karnofsky. We can roughly formulate the question as the one of how to split resources between worldviews that recommend very different courses of action. This kind of uncertainty, not about the claims of a certain theory, but about which of these theories we should choose, is similar to problems around moral uncertainty, like the ones discussed in Will MacAskill's thesis. What I will be doing in the following is to apply two of the methods used in that thesis to a concrete example of someone trying to figure out which actions to prioritise, given uncertainty as to which worldview is correct.

Worldviews, and where they diverge

One example that Holden discusses in the podcast is the comparison between helping humans and animals (alive today). So, for instance, we could help a human for $1,000 and a chicken for $1 and then face the decision which one to prioritise.

This comes down to questions of whether we value chickens, on what grounds we would value humans and not chickens, and how much exactly we should value chickens if we do value them. So, if you value chickens 10% as much as humans, you might want to decide in favour of chickens, and if you do not value chickens at all, helping the humans would be a much better decision.

We can think of these two outlooks as two different "worldviews". While I do not have a clear definition of that term (Holden himself says that the concept is "a very fuzzy idea"), relevant features of worldviews are:

  • Each yields particular outcomes (e.g. "prioritise X") that are very different from those of other worldviews.
  • These different outcomes might hinge on answering just one question differently (e.g. "Do chickens have moral value?"), or perhaps a few crucial questions.
  • Worldviews are constituted by a combination of views and beliefs that are not only normative, but also empirical, which makes it difficult to disentangle claims they make when we want to deal with uncertainty about worldviews (thanks to Max Daniel for pointing this out).

In the next section, I will discuss moral uncertainty, that is, uncertainties between first-order normative theories (such as Virtue Ethics, Hedonistic Utilitarianism, or Kantianism). This does not mean that I think that worldviews and normative theories are the same thing, even though I think we can apply measures of uncertainty to them in similar ways. I will also use the more neutral term "view", whenever I do not think it is important to distinguish the two terms.


The worldview split problem on an individual basis

Let's assume the fictitious graduate mentioned above (let's call her Lisa) has to pick a single option, for example in deciding which cause to devote her career to. When making this choice, it seems reasonable to not go for the option she likes most, but to also take into account how certain she is that each of the worldviews she derives the options from is correct.

This kind of thinking is what motivates the methods which Will uses in his thesis on normative uncertainty. He discusses examples in which agents have to weigh options which are ordered differently according to different normative theories, while being uncertain as to which of the theories is correct.

These decision-situations include a decision-maker at a specific point in time, who has at her disposal

  • a set of possible options (actions she can perform),
  • a set of theories that give an ordering of the options in terms of their choice-worthiness (whether, and sometimes, how much an option is preferred),
  • and a credence function that assigns each theory a subjective likelihood of being true.

This process is analogous to expected utility calculation, replacing "expected utility" with "expected choice-worthiness".

Below, I will apply these methods - ways of calculating and maximising expected choice-worthiness - to the case of different worldviews and see whether it is helpful in dealing with the kinds of "worldview split problems" discussed above.


Finding theories behind the worldviews

One major difficulty in assigning credences to worldviews is that often it is not clear what exactly constitutes a worldview. In this section, I explain why in the comparison further down, I am using the distinction between person-affecting and impersonal views to rank options. Feel free to skip to the next section if you don't want to hear all the details!

Person-affecting views capture the intuition that an act can be bad only if it is bad for someone, while impersonal views aim to maximize total (net) wellbeing.

The reason I searched for a different split in worldviews is that just attributing one worldview to each option made the whole prioritising framework uninformative: if our worldview is "I should care most about humans alive today" and one option is "do whatever is best for humans alive today", the framework's results will always be aligned with the worldview we assign highest credence to. If we assign this credence mainly based on intuition, we end up with what we would do if we just went for whatever we like best.

What I did to get around this was to note down some things someone might plausibly care about, and try to cluster them into more informative categories.

Things Lisa cares about:
  • Pursuing paths of action that produce good consequences.

  • Trying to cultivate care for others. This works well for other humans, somewhat for animals, but breaks down when we think about alien minds.

  • Capability to suffer as a very important aspect in who we assign moral value to.

  • Self-improvement: this includes virtues of good conduct in interaction with others, virtues like wisdom aimed at improving the way one makes decisions, and a general striving to become a better person. 

In addition, a flourishing world in which beings are happy, satisfy their preferences, and realise a variety of values seems like something worth caring about.

A lot of these values can be made use of in different worldviews: a commitment to bringing about the best possible consequences will vary according to which consequences are judged as best. A commitment to cultivating care will vary according to which beings we think of as ultimately "care-worthy" (other humans? animals? future beings?). For both, the person-affecting/impersonal distinction is relevant in deciding in which beings we should care about and which consequences we judge to be good. Capability to suffer seems like a less volatile criterion, which is uncertain still, but I'm hoping that this uncertainty is mainly empirical.

Looking at different virtues, there are some which which seem to favour a presentist perspective, centering around improving oneself and caring for others in the moment, and others which might favour long-termism (e.g. if we think that humanity as a whole is acting wisely by preventing its own extinction).

I have tried to show that just caring about producing good consequences or being virtuous does not give us an informative ranking of options, but that distinguishing between person-affecting and impersonal views does. I will apply this idea in the next section.


Applying this

Borda Rule

I started by ranking the options according to the Borda Rule, which Will uses in his thesis for theories that give merely ordinal choice-worthiness (that is, they give a preferred ranking, but do not provide information as to how much better or worse an option is) and where choice-worthiness is not comparable across theories.

Option person-affecting views (40% credence) impersonal views (60% credence)
(A) humans today 2 0
(B) animals today 1 1
(C) long-term future 0 2


The above table gives an ordering of preferences which seems roughly plausible to me, but might vary according to whether we take "person-affecting" to be about humans or including animals, and how we deal with the future. For example, we might think only about persons that already exist today, or additionally consider people who would have come into existence regardless of our intervention.

Multiplying the Borda score with the credence Lisa assigns to person-affecting and impersonal views, we get the following:

(A): 0.4*2+0.6*0= 0.8

(B): 0.4*1+0.6*1= 1

(C): 0.4*0+0.6*2= 1.2

We see that under this calculation, Option (C) - prioritizing the long-term future - scores highest. If we are very uncertain about which view is best and assign a 50% credence to each, we get:

(A): 0.5*2+0.5*0= 1

(B): 0.5*1+0.5*1= 1

(C): 0.5*0+0.5*2= 1

This makes it seem like, in this case, a merely ordinal ranking is still not better than just going with a simple intuition. If we add the assumption that choice-worthiness can be compared across different theories, and we think we can say something about how much more desirable one outcome is over the other, we can try our luck with cardinal ordering and see whether that is more informative.

Cardinal Ordering

For example, we might assign the following choice-worthiness to the different options under the two theories we are considering:

Option person-affecting views (40% credence) impersonal views (60% credence)
(A) humans today 0.7 0.05
(B) animals today 0.2 0.05
(C) long-term future 0.1 0.9


Credence-weighted, we get the following results across the two views:

(A) = 0.4*0.7+0.6*0.05= 0.31

(B) = 0.4*0.2+0.6*0.05= 0.11

(C) = 0.4*0.1+0.6*0.9= 0.58

Where do I get these numbers from? They are based on my intuition that under person-affecting views, humans alive today are considered as more important than either animals and the long-term future, while for impersonal views, the long-term future outweighs both current human and animal welfare by a lot. I was surprised to see such a low score assigned to animal welfare, since I intuitively think that animal welfare matters a lot. What the low score reflects is my (until now implicit!) assumption that person-affecting views really only deal with persons, excluding non-human animals. It also shows that I think person-affecting views only care about persons alive today, which is a strand of person-affecting views that differs from "caring only about people who would have existed anyway".

If we do think that there is some merit in the intuitive cardinal ordering I provided, this ordering still gives the same result for a 50-50 credence:

(A) = 0.5*0.7+0.5*0.05= 0.375

(B) = 0.5*0.2+0.5*0.05= 0.125

(C) = 0.5*0.1+0.5*0.9= 0.5

Option A would only be favoured if we assign 60% credence or more to person-affecting views being right (yielding a score of 0.44 versus option C scoring 0.42). This means that if Lisa gathered evidence that made her assign a credence of 60% or more to person-affecting views, she would have to not prioritise working on the long-term future, but on helping humans alive today.


I am aware of the assigned credences seeming somewhat random, in only reflecting the subjective likelihood that someone assigns to a certain view being true. I am also unsure about whether the orderings I ascribed to different views are correct interpretations of these views. However, I hope that having ascribed numbers to different orderings might facilitate discussion later on, making assumptions more tangible and thereby easier to challenge. The extent to which we should be bothered about this problem may also depend on what we would hope to use this method for.

As I mentioned above, worldviews encompass lots of different claims which are hard to disentangle. I tried to get around this by turning this into a question about person-affecting versus impersonal views, but very likely lost something along the way. This is why I hope that someone else will find this a useful prompt to propose a better way of thinking about this question.


Was this useful? /What did I learn? / What will I do as a result?

I am probably biased in favour of thinking that writing this post was useful because I invested time in doing so. Other reasons why this felt like a useful pursuit are that it was much more productive than just spending time reading on my own, since setting myself a deadline increased the amount I read and having a question in mind made my reading more focused and engaging.

As for other people, I hope this might provide a model that others can use to make explicit their thinking about their priorities and their uncertainties therein. Other benefits might lie in motivating people to engage with the question at large, or in prompting someone to see flaws in what I was doing and do it better in the future.

I learnt what my intuitive models of different moral theories look like and now have a much clearer grasp of where they are not as clear as I would hope them to be. I also enjoyed understanding what information might change my personal cause prioritisation. One immediate result of writing this is that I have a much better sense of which things I want to learn more about because doing so might affect my future behaviour.

If you have any answers to the questions and problems raised above, or anything else you would like to add, please do leave a comment!

Comments (6)

Comment author: MichaelPlant 16 May 2018 02:53:44PM 3 points [-]

Hello Ronja. I think it would be helpful to know if you think this is different from other approaches to moral uncertainty and, if so, which ones. I don't know if you take yourself to be doing something novel or providing an example using an existing theory.

Comment author: Ronja_Lutz 16 May 2018 04:54:19PM 4 points [-]

Hi, thanks for your comment :) Seems like I should have made that clearer! Since what I'm doing is applying Will's approach, the approach is not itself new. I haven't seen it discussed with regards to the worldview-split problem, but since I ended up condensing different "worldviews" into a decision between two theories, it turned out to be basically the same (which is not without problem, for that matter). I still found it valuable to try out this process in practice, and since I am expecting many people to not have read Will's thesis, I hoped this would provide them with an example of such a process. One person told me they found it valuable to use this way of thinking for themselves, and someone else said they were more inclined to read the actual thesis now, so I think there is some value in this article, and the issue might be more about the way I'm framing it. If you have an idea for a framing you would have found more useful, I'd be happy to know. Do you think just adding a sentence or two at the start of the article might do?

Comment author: MichaelPlant 17 May 2018 03:39:34PM 2 points [-]

Yeah, I think it would be good to put the research in context - true for posts here as other pieces of work - so readers know what sort of hat they should be wearing and if this is relevant for them.

Comment author: kbog  (EA Profile) 16 May 2018 04:02:18PM *  1 point [-]

So I skimmed this and it looks like you are basically just applying MacAskill's method. Did I miss something?

Btw, whether to assign ordinal or cardinal scores to things isn't really something that you should do in the context of normative uncertainty. It should come from the moral theory itself, and not be altered by considerations of uncertainty. If the moral theory has properties that allow us to model it with a cardinal ranking, then we do that, and if it doesn't then we use an ordinal ranking. One moral theory may have ordinal rankings and another may have cardinal ones. By the way, as far as MEC is concerned, an ordinal moral ranking is just a special case of cardinal moral rankings where the differences between consecutively ranked options are uniform.

Comment author: Denkenberger 20 May 2018 07:59:23PM 1 point [-]

Interesting - so then interventions that do well on both long-term future and humans today like AI and alternate foods would do very well by your numbers.

Comment author: Andaro 30 May 2018 02:13:39PM 0 points [-]

I used to think of this in terms of sanity heuristics. For example, if you consider an altruistic act A relative to another possible altruistic act B, check if A would cause more victimization of innocent sentient entities, e.g. rape, torture or similar nonconsensual violations. Measured by number and/or severity. If so, choose B instead and sanity-check against C, wash rinse repeat until you're at least indifferent between the choices on this dimension. A kind of "do no harm" or rather "don't increase evil" principle. The problem is that this is incompatible with those forms of x-risk reduction that focus on making the world bigger rather than better. And that in turn got me a lot of flak from the EA community.

I wrote "used to think" because I gave up on altruism altogether since then. From a purely self-interested POV, there's no reason to apply any such sanity heuristic to my choices. The hard truth is, the disutility of others is not my disutility. That's a straightforward fact of life. So why should I treat it as such? It's really hard to justify on logical grounds.

My axiomatic disagreements with high-status "altruists" who all but trademarked altruism itself, e.g. Elon Musk and the EA community, actually helped me in this reflection process. To see how badly those altruists went wrong (from my perspective) and how much they compromised their own resource base and wellbeing while causing more innocent victims than they prevent opened my eyes and led me back to the most fundamental question, "Why care in the first place?"

I still do instrumental altruism, but it is strictly limited to facilitating the interests of those who facilitate my personal interests. Similarly, I undermine the interests of those who undermine my personal interests.

My only disagreement with EA now is when you harm the overall interests of people in the countries where you apply your lobbying and propaganda efforts. For example, I don't want more costly immigration to my country, or unilateral GHG reductions, or higher consumer prices to save the rainforests or for animal welfare, or tax funds going to scientific projects that don't facilitate my wellbeing or the competitiveness of my allies. I'm willing to penalize all such efforts that make me personally worse off severely.

This is not limited to EA, of course, and you're not the worst offenders. Organized religion is actually at the top of my enemy list for now. They've been working ridiculously hard to harm my personal interests to an almost comical degree. For example, through lobbying and propaganda efforts to reduce my personal end-of-life choice set without my consent. As a consequence, I now routinely shift all my indifference curves to harm all their interests and values as much as possible by all means available, whenever applicable. I don't think they had and idea what they were doing, either. When I discussed with them, I always had the impression that they thought undermining the personal interests of other people is somehow a free action. Like, if they use moralistic language or refer to the will of their God, this somehow works like a magic spell that will make the retaliation damage go away. Of course, none of that is true.

The only remotely good thing about them from my perspective is their pedophilia. I'd like to see consensual pedophilia normalized, since I would personally welcome this increase in my sexual choice set. I would love to live in a world where it is legal and socially accepted to buy consensually made child porn or pay willing 12-year-olds for blowjobs during the holidays. They could make an honest buck, and everybody would gain more sexual options. So I'm actually not upset about all those pedophilic priests. It's good to see them come out of their closet. Unfortunately, there's never any real distinction made between consensual conduct and child sexual abuse. I can't even tell from the media reports if the kids molested by organized religion had an honest exit option explained to them or if they were threatened, lied to, or otherwise coerced. Overall the pedo bashing has gotten worse rather than better over the last few years, and evil opportunists like Ross Douthat are trying to leverage the "me too" phenomenon to call for bans on consensually made adult (!) porn. Ridiculously stupid enemy action.

Anyway, sorry for the long post; I just wanted to let you know that I apologize for my earlier posts that suggested increasing x-risk as an altruistic strategy. They were motivated by the benevolent concern that lower x-risk will increase the total amount of evil in the universe. Which of course it actually will. Since then, I gave up on benevolence altogether, with the exception of rewarding those who facilitate my personal interests anyway. I don't really care what happens after my death, because there's no good reason for me to care. As long as you don't undermine the rational self-interest of people in the West or my person specifically, I have no more problem with EA. Just don't parasitize us without giving us higher value back, economically or in terms of our rights and liberties; others are already paying a huge price for trying that.