turchin comments on Nothing Wrong With AI Weapons - Effective Altruism Forum

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (22)

You are viewing a single comment's thread.

Comment author: turchin 01 November 2017 09:12:46PM *  0 points [-]

Thanks again for the interesting post. After rereading I have some more thoughts on the topic.

I would add that LAW is not the same as Military AI, and LAW as the safest part of the military AI. M.Maas showed that Military AI consists of the several layers, where LAWs are on the lowest. https://hcss.nl/report/artificial-intelligence-and-future-defense

An advanced Military AI will probably include several other functions (some already exist):

1.Strategic planning of winning in war

2.Direct control of all units inside the country's defence systems, which may include drones, ships, nuclear weapons, humans, and other large and small units

3Nuclear deterrence part, which consists of the early warning system and dead hand second strike system.

4Manufacturing and constructing new advanced weapons

5Cyberweapons, that is instruments "to elect Trump" or to turn off adversaries' AI or other critical infrastructure.

Each of this 5 levels could have a global catastrophic failure, even without starting uncontrollable self-improving.

1.Strategic planning may have superhuman winning ability (think about AlphaGo Zero, but used as general) or could have a failure if it suggests "to strike first now or lose forever",

2 Global army controlling system could propagate a wrong command.

3 The Early warning system could create false alarm (had happened before). there also could be flash-crash stile unexpected war between two Military AIs of two adversarial nation states.

4Weapons manufacturing AI may be unexpectedly effective in creating very dangerous weapons, which later will be used with global consequences, more severe than nuclear war.

5Use of cyberweapons also may be regarded as an act of war or help to elect a dangerously unstable president (some think that this already happened with DT). Cyberwar may also affect other's side critical infrastructure or rewrite other's side AI goal function, which is bad outcomes.