Hide table of contents

Daniel Kokotajlo, Diego Caleiro, Ramana Kumar and I recently discussed the idea of Permanent societal improvement - non-Xrisk related ways of affecting, and hopefully improving, the far future. These are actions we can take now that will have some multiplicative effect on the value of the future of humanity. This post is intended as the beginning of a conversation, not the end of a research project, and we eagerly await feedback and more ideas. Also please bear in mind that not everyone agreed with all the ideas, and any mistakes remain my own.

 

A toy model:


Suppose there is a 5% chance humanity will be destroyed in 2100, and if we survive that great filter then we will go on to colonise the light cone. Assuming this is a ‘good’ colonisation, full of happy, enlightened, virtuous people, it seems that reducing Existential Risk by 5% to 0% would roughly increase the Expected Value of the future by 5.3%. We could compare this to an action that would make the colonised universe 10% better - this would then increase the EV of the future by roughly 10%. So improving the future, in this toy example, could be dramatically better than reducing Xrisk.

 

What types of things could be Permanent Societal Improvements?

 

A major restriction is that they have to be things that would not otherwise be done later. If I invent something that would otherwise have been invented 20 years later, I have only improved the world by ( 20 years x impact of invention ) , not ( lifespan of humanity x impact of invention ).* This is quite a strong restriction on what could count as such permanent improvements.

 

Here are a few broad categories we came up with.

 

Influencing Lock-in

  • It is possible that humanity will end up ‘locked in’ to a certain political state. This could be a Singleton - an agent with complete control, either AI or totalitarian state or a stable multipolar society, maybe due to EM competition.

  • If this is the case, then affecting which political state humanity gets locked into would have a permanent effect on the future.

  • Alternatively, we might affect what the future cared about. Under some types of Singleton, virtually any ethical debate could become a very pressing issue: we want to make sure the right side is preserved and propagated. Maybe it is important to persuade people that animals are morally valuable now so that an AI will care about them (though perhaps CEV obviates the need for this.) Or maybe we need to make sure the future Hegemon cares enough about art so the lightcone isn’t deprived of it.

  • Value Lock-in could happen even without a Singleton, given some new technologies or memes. For example, if we invent memetic or technological ways to reinforce existing values, (like brainwashing but more effective) then existing value systems could become significantly more entrenched.

 

Compounding resource constraints

  • If there is some resource which is going to be a constraint on the growth of moral value, we could affect the future by investing now to reduce the constraint. This is especially true if the resource exhibits compound growth.

  • For example, if total population were to grow at 1% a year indefinitely, by having some extra children now (say 1%) we could permanently increase the future population in a multiplicative way. If you subscribe to an aggregative theory of ethics, this could make the future 1% more valuable.

  • If the future will be bounded by the speed of light, launching ships now could help alleviate the volume constraint.
  • Or maybe we could invest in server capacity in readiness of a EM future.

 

Moral Progress / Decay

  • If humanity makes moral progress, we might improve the future by accelerating this process.

  • Alternatively, if humanity suffers from value drift, we could improve the future by retarding this decay.

  • The benefit of accelerating moral progress is much less if there is an ideal ethics we are converging towards, as then we only get the benefit from accelerating near-perfect ethics.

  • Conversely, if our values are drifting in such a way that most of the future will be of no value, perhaps due to value fragility, then delaying the decay could dramatically improve the future. If we suffer 1% drift a year, then standing athwart history for a year would improve the total value of the future by 1%.

 

Original Sin

  • Some people think that the British Empire was permanently ‘tainted’, in some way, by its early endorsement of slavery, and that this moral taint persisted even after it had abolished slavery in most of the world. If this is true, it could be valuable to ensure the future isn’t founded in some way that permanently taints it.

  • Conversely, maybe having some people from the 20th and 21st centuries could be a long-lasting source of pride and joy for future generations, so some cryonics could be considered a permanent improvement. Similar things could be said about historical artifacts, beautiful natural landmarks, etc.

 

Coordination problems

  • If humanity colonises the stars we may end up being fractured by distance, different colonies unable to communicate. Perhaps when the first ships are sent a lasting convention could be established that all colonies should send updates about their history back to earth. This history could be an object of great value, but establishing this convention might be something that could only be done very early on in the colonisation process.

  • Alternatively, we could establish property right norms to divide up the lightcone and prevent conflict. By establishing a norm now that colonised could claim whatever they wanted if they traveled directly away from earth, but could not ‘cross-colonise’ into other sectors, we could prevent future wars. This norm would be much harder to establish once the earth was no longer the clear schelling point for the origin, and once it was clear who the ex-post winners and losers from this policy would be.

  • Establishing norms that will protect biological humans and EMs from Hansonian competition - like a right to retire.

  • If uploads are not conscious, it might be important to agree on this before EMs massively outnumber biological humans; after that point it would become much harder.

 

* ignoring whatever else the future would-be inventor would otherwise do with their resources.

 

Comments10


Sorted by Click to highlight new comments since:

Or maybe we could invest in server capacity in readiness of a EM future.

This one seemed out of place to me. Conditioned on the time we start expanding and the rate at which we expand, we're going to have access to some fixed set of resources at a given point in the future, so I don't see how investing in server capacity now affects our server capacity in the far future. (though I do agree that affecting the start time and rate of expansion could be permanent improvements.)

Establishing norms that will protect biological humans and EMs from Hansonian competition - like a right to retire. If uploads are not conscious, it might be important to agree on this before EMs massively outnumber biological humans; after that point it would become much harder.

These seem to be about simply picking the right policies now and locking them in. It might also be important to lock in the right policies vis-a-vis privacy, the death penalty, property rights, etc etc, but why should we think that we can lock such policies in now? This reduces to either "minimize value drift" or "create a singleton", both of which I agree with but you already listed them.

Have you seen Nick Beckstead's slides on 'How to compare broad and targeted attempts to shape the far future'?

He gives a lot of ideas for broad interventions, along with ways of thinking about them.

So we get astronomical stakes by multiplying a large amount of time by a large amount of space to get a large light cone of potential future value. Interventions that work along only one of those dimensions -- say, I bury a single computer that generates one utilon per year deep underground, which continues to run for the life of the universe, or I somehow grant a one-off one utilon to every human alive in the year 1 billion -- are dominated by those interventions that affect the product of space and time (e.g. the interventions you listed here). But if there were just one more dimension to multiply, then interventions that addressed the product of all three might dominate all considerations that we currently think about.

Yep. Any ideas what such an other dimension might be? (There are of course the "normal" other dimensions, like average well-being, that are included in the calculation of utilons.)

"Assuming this is a ‘good’ colonisation, full of happy, enlightened, virtuous people, it seems that reducing Existential Risk by 5% to 0% would roughly increase the Expected Value of the future by 5.3%."

How did you get 5.3%?

(100/95) - 1

An important topic!

Potentially influencing lock-in is certainly among my motivations for wanting to work on AI friendliness, and doing things that could have a positive impact of a potential lock-in has a lot speaking for it I think (and many of these things, such as improving the morality of the general populous, or creating tools or initiatives for thinking better about such questions, are things that could have significant positive effects also if no lock-in occurs).

As to example of having-more-children out of far-future concerns, I think this could go the other way also (although I don't necessarily thing that it would - I really don't know). If we e.g. reach a solution where it is decided that all humans have certain rights, can reproduce, etc, but also decide that all or a fraction of the matter in the universe we have little need for are used to increase utility in more efficient ways (e.g. by creating utilitronium or by creating non-human sentient beings with positive and meaningful existences), then a larger human population could lead to less of that.

A major restriction is that they have to be things that would not otherwise be done later. If I invent something that would otherwise have been invented 20 years later, I have only improved the world by ( 20 years x impact of invention ) , not ( lifespan of humanity x impact of invention ).* This is quite a strong restriction on what could count as such permanent improvements.

Hasn't Bostrom said something about how delays in technological progress create astronomical losses because they slow down subsequent technology?

Along the same lines I think economic progress and setting norms in international law could similarly have compounding effects that affect the far future pretty well. Otherwise I'm not sure what really can be done about these things besides raising awareness and spreading of ideas.

[anonymous]-2
0
0

You seem to want to look at oughts while not knowing what is. Oughts are limited to the possible and knowing what is possible can only happen by understanding what is and why it is. Understanding human nature then is the first critical task, and to do that requires a good understanding of evolution since humans evolved. This has been my work for several decades and I am still hoping that folks in your movement will see value in it instead of wandering in a vast intellectual wasteland wearing blindfolds and talking about how great it would be if you could direct humanity to the Great Lakes.

I not sure I understand your argument. Could you help me out with some examples of:

  • Effective Altruists "wandering in a vast intellectual wasteland wearing blindfolds and talking about how great it would be if you could direct humanity to the Great Lakes"
  • How an understanding of human evolution would help us to find out what we ought to do.
Curated and popular this week
 ·  · 25m read
 · 
Epistemic status: This post — the result of a loosely timeboxed ~2-day sprint[1] — is more like “research notes with rough takes” than “report with solid answers.” You should interpret the things we say as best guesses, and not give them much more weight than that. Summary There’s been some discussion of what “transformative AI may arrive soon” might mean for animal advocates. After a very shallow review, we’ve tentatively concluded that radical changes to the animal welfare (AW) field are not yet warranted. In particular: * Some ideas in this space seem fairly promising, but in the “maybe a researcher should look into this” stage, rather than “shovel-ready” * We’re skeptical of the case for most speculative “TAI<>AW” projects * We think the most common version of this argument underrates how radically weird post-“transformative”-AI worlds would be, and how much this harms our ability to predict the longer-run effects of interventions available to us today. Without specific reasons to believe that an intervention is especially robust,[2] we think it’s best to discount its expected value to ~zero. Here’s a brief overview of our (tentative!) actionable takes on this question[3]: ✅ Some things we recommend❌ Some things we don’t recommend * Dedicating some amount of (ongoing) attention to the possibility of “AW lock ins”[4]  * Pursuing other exploratory research on what transformative AI might mean for animals & how to help (we’re unconvinced by most existing proposals, but many of these ideas have received <1 month of research effort from everyone in the space combined — it would be unsurprising if even just a few months of effort turned up better ideas) * Investing in highly “flexible” capacity for advancing animal interests in AI-transformed worlds * Trying to use AI for near-term animal welfare work, and fundraising from donors who have invested in AI * Heavily discounting “normal” interventions that take 10+ years to help animals * “Rowing” on na
 ·  · 3m read
 · 
About the program Hi! We’re Chana and Aric, from the new 80,000 Hours video program. For over a decade, 80,000 Hours has been talking about the world’s most pressing problems in newsletters, articles and many extremely lengthy podcasts. But today’s world calls for video, so we’ve started a video program[1], and we’re so excited to tell you about it! 80,000 Hours is launching AI in Context, a new YouTube channel hosted by Aric Floyd. Together with associated Instagram and TikTok accounts, the channel will aim to inform, entertain, and energize with a mix of long and shortform videos about the risks of transformative AI, and what people can do about them. [Chana has also been experimenting with making shortform videos, which you can check out here; we’re still deciding on what form her content creation will take] We hope to bring our own personalities and perspectives on these issues, alongside humor, earnestness, and nuance. We want to help people make sense of the world we're in and think about what role they might play in the upcoming years of potentially rapid change. Our first long-form video For our first long-form video, we decided to explore AI Futures Project’s AI 2027 scenario (which has been widely discussed on the Forum). It combines quantitative forecasting and storytelling to depict a possible future that might include human extinction, or in a better outcome, “merely” an unprecedented concentration of power. Why? We wanted to start our new channel with a compelling story that viewers can sink their teeth into, and that a wide audience would have reason to watch, even if they don’t yet know who we are or trust our viewpoints yet. (We think a video about “Why AI might pose an existential risk”, for example, might depend more on pre-existing trust to succeed.) We also saw this as an opportunity to tell the world about the ideas and people that have for years been anticipating the progress and dangers of AI (that’s many of you!), and invite the br
 ·  · 12m read
 · 
I donated my left kidney to a stranger on April 9, 2024, inspired by my dear friend @Quinn Dougherty (who was inspired by @Scott Alexander, who was inspired by @Dylan Matthews). By the time I woke up after surgery, it was on its way to San Francisco. When my recipient woke up later that same day, they felt better than when they went under. I'm going to talk about one complication and one consequence of my donation, but I want to be clear from the get: I would do it again in a heartbeat. Correction: Quinn actually donated in April 2023, before Scott’s donation. He wasn’t aware that Scott was planning to donate at the time. The original seed came from Dylan's Vox article, then conversations in the EA Corner Discord, and it's Josh Morrison who gets credit for ultimately helping him decide to donate. Thanks Quinn! I met Quinn at an EA picnic in Brooklyn and he was wearing a shirt that I remembered as saying "I donated my kidney to a stranger and I didn't even get this t-shirt." It actually said "and all I got was this t-shirt," which isn't as funny. I went home and immediately submitted a form on the National Kidney Registry website. The worst that could happen is I'd get some blood tests and find out I have elevated risk of kidney disease, for free.[1] I got through the blood tests and started actually thinking about whether to do this. I read a lot of arguments, against as well as for. The biggest risk factor for me seemed like the heightened risk of pre-eclampsia[2], but since I live in a developed country, this is not a huge deal. I am planning to have children. We'll just keep an eye on my blood pressure and medicate if necessary. The arguments against kidney donation seemed to center around this idea of preserving the sanctity or integrity of the human body: If you're going to pierce the sacred periderm of the skin, you should only do it to fix something in you. (That's a pretty good heuristic most of the time, but we make exceptions to give blood and get pier