Hide table of contents

This is a linkpost for rethinkpriorities.org/longtermism-research-notes/examining-pathways-through-which-narrow-ai-systems-might-increase-the-likelihood-of-nuclear-war

 

The goal of this post is to sketch out potential research projects about the possible pathways through which narrow AI systems might increase the likelihood of nuclear war.[1]

The post is mostly based on reading that I did in February-March 2022. I wrote the basic ideas in August 2022, and made quick edits in May 2023, largely in response to comments that I received on an unpublished draft. It is now very unlikely that I will do more research based on the ideas here. So, I am sharing the sketch with the hope that it might be helpful to other people doing research to reduce risks from nuclear weapons and/or AI.[2]

Since my research here is mostly a year old, I unfortunately do not engage here with the newest work about the topic.[3] Indeed, I expect that some or many of the points that I make here will have been made in recent published work by others. I’m sharing my notes anyway, however, since it seems quick to do so, and in case my notes are able to add any additional value to the existing literature — e.g. by highlighting potential future research projects. 

 

Please note that this is a blog post, not a research report, meaning that it was produced quickly and is not to Rethink Priorities’ typical standards of substantiveness, careful checking for accuracy, and reasoning transparency.

Please also note that many topics in nuclear security are sensitive, and so people working on this area should be cautious about what they publish. Before publishing, I ran this post by a couple of people whom I expect to be knowledgeable and careful about infohazards related to nuclear security. If you do research based on this post and are not sure whether your work would be fine to share, please feel free to reach out to me and I’d be happy to give some thoughts.

 

Summary

There seem to be four main possible pathways discussed in existing literature by which narrow AI systems might increase nuclear risk. I consider each in turn:

  1. How much additional nuclear risk is there from reduced second-strike capability? This refers to narrow AI systems reducing the ability of a country to launch a “second” nuclear strike as retaliation against a “first” nuclear strike against that country by someone else. This might increase the likelihood of nuclear war by weakening nuclear deterrence.
  2. How much risk from delegating nuclear launch authority? In this pathway, decisions about launching nuclear weapons are (partly) delegated to AI. Intrinsic weaknesses with current AI approaches — in particular brittleness — mean that the AI may launch when we would not want it to.[4]
  3. How much risk from a degraded information environment? In this pathway, narrow AI technologies reduce the ability for leaders to make informed decisions about nuclear launches. This may increase the likelihood that leaders launch nuclear weapons when they would not do so if they had a full picture of what’s going on.
  4. How much risk from lethal autonomous weapons (LAWs)? In this pathway, there is an incident with conventionally-armed lethal autonomous weapons that escalates to nuclear war. Alternatively, LAWs change the balance of conventional military power, making the countries that are now much weaker more tempted to resort to using nuclear weapons.

 

For each pathway, I list some questions that would need to be answered to understand how plausible/big the effect is. I sometimes also give brief arguments for why I think the expected effect is overrated (at least by the people who propose it).

 

Click here for the full version of this post on the Rethink Priorities website.

 

Acknowledgments

This blog post is a project of Rethink Priorities. It was written by Oliver Guest.

Thank you to Matthijs Maas for originally pointing me towards a lot of helpful existing work, and to Patrick Levermore for feedback on an earlier draft of this post. They do not necessarily endorse the points that I make here.

 

If you are interested in RP’s work, please visit our research database and subscribe to our newsletter.

 

  1. ^

     The project could be expanded to include ways in which narrow AI systems might decrease the likelihood of nuclear war. See, for example, this journal article.

  2. ^

     Because I am sharing quickly, this post unfortunately has more jargon and unreferenced empirical claims than I would like.

  3. ^

     Recent research that I am aware of, but have unfortunately not had much opportunity to engage with, include:

    Maas, M. M., Matteucci, K., & Cooke, D. (2022). Military Artificial Intelligence as Contributor to Global Catastrophic Risk. SSRN Electronic Journal. https://doi.org/10.2139/ssrn.4115010 (Forum post here)

    Nadibaidze, A., & Miotto, N. (2023). The Impact of AI on Strategic Stability is What States Make of It: Comparing US and Russian Discourses. Journal for Peace and Nuclear Disarmament, 1–21. https://doi.org/10.1080/25751654.2023.2205552 

    Rautenbach, P. (2022, November 18). Artificial Intelligence and Nuclear Command, Control, & Communications: The Risks of Integration. EA Forum. https://forum.effectivealtruism.org/posts/BGFk3fZF36i7kpwWM/artificial-intelligence-and-nuclear-command-control-and-1 

    Ruhl, C. (2022). Autonomous weapons systems & military AI. Founders Pledge. https://founderspledge.com/stories/autonomous-weapon-systems-and-military-artificial-intelligence-ai (Forum post here)

  4. ^

     I use “brittleness” to mean an AI system's inability to adapt to novel situations or handle edge cases it has not encountered during training.

8

0
0

Reactions

0
0

More posts like this

Comments2
Sorted by Click to highlight new comments since:

oeg - this is an important and apparently neglected topic.

Too many people seem to think that extinction risk only arises with AGI, and they neglect the many ways that near-term narrow AI can amplify other X-risks. I think you're correct that narrow AI can amplify nuclear war risks.

I would suggest a couple of other ways this could happen:

  1. small, autonomous drones could greatly increase the risk of assassinations of major nation-state leaders, esp. if the drones have some AI face-recognition abilities, long-term loitering abilities (e.g. lying in wait for ideal opportunities), and tactical planning abilities (to evade security forces counter-measures). (e.g. see the book 'Swarm Troopers' by David Hambling). This isn't a matter of large LAWS swarms escalating conventional conflict; rather, it offers the opportunity to terrorists, dissidents, separatists, partisans, rebels, etc to eliminate heads of state they consider bad -- but from a great distance in time and space, with a high degree of plausible deniability. And it opens the possibility of highly manipulative false flag operations.
  2. AI-enabled deepfake videos could portray heads of state or military leaders (falsely) announcing escalations of conventional or nuclear strikes, which could lead their enemies to counter-escalate; the information environment could become so corrupted and unreliable that the likelihood of making strategic errors under the 'fog of war' might be greatly increased.

It seems like in the full post you didn't reference or cite this RAND report, which is directly relevant; https://www.rand.org/pubs/perspectives/PE296.html

More from oeg
Curated and popular this week
Relevant opportunities