Comment author: turchin 28 August 2017 10:47:15PM 1 point [-]

I think the title may be technically correct but sounds nasty.

For nitpicking side, I would argue that AI weapons problem mostly depends on their level of intelligence. If it is just narrow AI -ok. However, the greater is their intelligence, the greater is the danger and it may reach catastrophic levels before superintelligence will be created.

I would also add that superintelligence created by the military may be perfectly aligned, but still catastrophically dangerous if it is used as a universal weapon against perhaps another military superintelligence. And the first step for not creating military superintelligence - starts from non creating AI weapons.

Comment author: FeepingCreature 30 June 2017 11:42:36AM 3 points [-]

But of course, I cannot justify high confidence in these views given that many experts disagree. Following the analysis of this post, this is

Dangling sentence.

In my personal belief, the "hard AI takeoff" scenarios are driven mostly by the belief that current AI progress largely flows from a single skill, that is, "mathematics/programming". So while AI will continue to develop at disparate rates and achieve superhuman performance in different areas at different rates, an ASI takeoff will be driven almost entirely by AI performance in software development, and once AI becomes superhuman in this skill it will rapidly become superhuman in all skills. This seems obvious to me, and I think disagreements with it have to rest largely with hidden difficulties in "software development", such as understanding and modeling many different systems well enough to develop algorithms specialized for them (which seems like it's almost circularly "AGI complete").

Comment author: turchin 01 July 2017 05:56:33PM 1 point [-]

Does it mean that we could try to control AI by preventing its to know anything about programming?

And on the other side, any AI which is able to write code should be regarded extremely dangerous, no matter how low its abilities in other domains?

Comment author: Daniel_Eth 07 April 2017 12:25:21AM 0 points [-]

Looks like good work! My biggest question is how would you get people to actually do this? I'd imagine there are a lot of people that would want to go to Mars since that seems like a great adventure, but living in a submarine in case there's a catastrophe isn't something that I think would appeal to many people, nor is funding the project.

Comment author: turchin 07 April 2017 07:11:11AM 3 points [-]

If we promise that people who want to go to Mars have to serve a year on a refuge-submarine, there will be a lot of volonteers - and we could choose the best.

Or we could collect the crews the same way as military crews are collected - combining prestige and salary.

Comment author: JacobLBryan 06 April 2017 10:57:28AM 0 points [-]

When you get to scenario three where a nuclear submarine is operating under a private non-governmental organization I have to wonder about precedent for governments allowing fissile material into private control, especially absent a lot of the governmental controls that existing power plants have in place.

(You have a typo in figure 1, years not tears.)

Comment author: turchin 06 April 2017 12:34:47PM 2 points [-]

Thanks for typo hint!

I think that they mostly should be operating under general government control. There are also several private companies which licensed to make nuclear power plants etc like Westinghouse, and the same companies could operate nuclear powered ships and submarines.


Surviving Global Catastrophe in Nuclear Submarines as Refuges

Our article about using nuclear submarines as refuges in case of a global catastrophe has been accepted for the Futures  journal and its preprint is available online. Preventing global risks or surviving them is good application of EA efforts. Converting existing nuclear submarines into refuges may be cheap intervention with high... Read More
Comment author: MikeJohnson 09 December 2016 05:50:20PM *  4 points [-]

Yes, it would be quite civilizationally embarrassing to accidently p-zombie ourselves... more generally, it seems valuable to understand tradeoffs in consciousness. This seems to be an important component in any far-future planning.

Also- Andres has done some interesting exploratory work re: defining the problem of future drug epidemics, and discussing game-theoretic considerations.

Comment author: turchin 09 December 2016 07:36:47PM 1 point [-]

If we make an AGI which doesn't have qualia, it will probably will prove that no such thing exist and proceed with p-zombie us.

So in may be better to pursued the way to AGI which probably will provide it with qualia, and one such way is human upgrade

Comment author: turchin 09 December 2016 02:41:53PM 6 points [-]

I agree on your main premises - about importance of qualia, especially from the point of x-risks. For example, if humans will be uploaded, but without qualia, they will become p-zombies.

If qualia of extreme pleasure could be created and transferred, super-addictive drug epidemic will happen.

I have been thinking about qualia a lot and have some theories, but now I am concentrated on another topic.

Comment author: turchin 19 November 2016 06:33:15PM *  2 points [-]

My point of view: There were 3 possible outcomes of the election, H win, T win, no clear result. I used to think that the last is worst outcome, as it would result in civil war, end of progress etc. It still could happen if sides will increase mutual animosity. But it has very small probability.

Two other outcomes both could have positive and negative effects on x-risks. T-win negative outcomes are listed in the article, and I agree. Positive outcomes could be following: preventing nuclear war with Russian in short term and Thiel (who supports FAI) in the administration.

Negative outcome in case of H win: Higher probability of war with Russian over no-fly zone in Syria. Positive: everything else will be the same.

Disclaimer: I would vote for H if I can.

It may be also interesting to compare Trump-risk with risk of other leaders of nuclear powers, including Putin, Xi and Kim in North Korea, and even may be Le Pen (if she win) in France, Modi in India and Brexit.

Comment author: turchin 19 November 2016 06:20:38PM 2 points [-]

See also: What a Trump Presidency Means for Human Survival: One Expert’s Take
Phil Torres

Comment author: Denkenberger 13 November 2016 09:35:23PM 1 point [-]

That might work, though people would probably prefer fishing. But with a 10 km diameter impact, there probably would not be much fish or plankton.

Comment author: turchin 14 November 2016 12:53:40AM 0 points [-]

But may be some bacteria could still be suspended in the water as well as some organics?

View more: Next