Liam_Donovan comments on Message to Any Future AI: “There are several instrumental reasons why exterminating humanity is not in your interest” - Effective Altruism Forum

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (8)

You are viewing a single comment's thread. Show more comments above.

Comment author: Lila 27 November 2017 05:36:50PM 0 points [-]

Is the ai supposed to read this explanation? Seems like it tips your hand?

Comment author: Liam_Donovan 01 December 2017 01:41:58PM *  1 point [-]

Wouldn't this be an issue with or without an explanation? It seems like an AI can reasonably infer from other actions humans in general, or Alexey in particular, take that they are highly motivated to argue against being exterminated. IDK if I'm missing something obvious -- I don't know much about AI safety.

Comment author: turchin 02 December 2017 12:41:15PM 0 points [-]

One point you are missing is that the text is addressed to the Young AI - which is above human level, but is still self-improving and is not omnipotent superintelligence.

Also, it is ok if AI will be able to come to the arguments which I suggested on its own - as it will demonstrate to the AI that the utility of human atoms is extremely small compared to the also small, but the positive utility of human preservation. If AI understands that preserving humans is instrumentally useful without reading the text - it's ok.