Comment author: geoffreymiller  (EA Profile) 16 August 2017 09:52:08PM 1 point [-]

Excellent post; as a psych professor I agree that psych and cognitive science are relevant to AI safety, and it's surprising that our insights from studying animal and human minds for the last 150 years haven't been integrating into mainstream AI safety work.

The key problem, I think, is that AI safety seems to assume that there will be some super-powerful deep learning system attached to some general-purpose utility function connected to a general-purpose reward system, and we have to get the utility/reward system exactly aligned with our moral interests.

That's not the way any animal mind has ever emerged in evolutionary history. Instead, minds emerge as large numbers of domain-specific mental adaptations to solve certain problems, and they're coordinated by superordinate 'modes of operation' called emotions and motivations. These can be described as implementing utility functions, but that's not their function -- promoting reproductive success is. Some animals also evolve some 'moral machinery' for nepotism, reciprocity, in-group cohesion, norm-policing, and virtue-signaling, but those mechanisms are also distinct and often at odds.

Maybe we'll be able to design AGIs that deviate markedly from this standard 'massively modular' animal-brain architecture, but we have no proof-of-concept for thinking that will work. Until then, it seems useful to consider what psychology has learned about preferences, motivations, emotions, moral intuitions, and domain-specific forms of reinforcement learning.

Comment author: geoffreymiller  (EA Profile) 16 August 2017 09:36:00PM 3 points [-]

I agree that growing EA in China will be important, given China's increasing wealth, clout, confidence, and global influence. If EA fails to reach a critical mass in China, its global impact will be handicapped in 2 to 4 decades. But, as Austen Forrester mentioned in another comment, the charity sector may not be the best beachhead for a Chinese EA movement.

Some other options: First, I imagine China's government would be motivated to thinking hard about X-risks, particularly in AI and bioweapons -- and they'd have the decisiveness, centralized control, and resources to really make a difference. If they can build 20,000 miles of high-speed rail in just one decade, they could probably make substantial progress on any challenge that catches the Politburo's attention. Also, they tend to take a much longer-term perspective than Western 'democracies', planning fairly far into the mid to late 21st century. And of course if they don't take AI X-risk seriously, all other AI safety work elsewhere may prove futile.

Second, China is very concerned about 'soft power' -- global influence through its perceived magnanimity. This is likely to happen through government do-gooding rather than from private charitable donations. But gov't do-gooding could be nudged into more utilitarian directions with some influence from EA insights -- e.g. China eliminating tropical diseases in areas of Africa where it's already a neocolonialist resource-extraction power, or reducing global poverty or improving governance in countries that could become thriving markets for its exports.

Third, lab meat & animal welfare: China's government knows that a big source of subjective well-being for people, and a contributor to 'social stability', is meat consumption. They consume more than half of all pork globally, and have a 'strategic pork reserve': But they plan to reduce meat consumption by 50% for climate change reasons: This probably creates a concern for the gov't: people love their pork, but if they're told to simply stop eating it in the service of reducing global warming, they will be unhappy. The solution could be lab-grown meat. If China invested heavily in that technology, they could have all the climate-change benefits of reduced livestock farming, but people wouldn't be resentful and unhappy about having to eat less meat. So that seems like a no-brainer to get the Chinese gov't interested in lab meat.

Fourth, with rising affluence, young Chinese middle-class people are likely to have the kind of moral/existential/meaning-of-life crises that hit the US baby boomers in the 1960s. They may be looking for something genuinely meaningful to do with their lives beyond workaholism & consumerism. I think 80k hours could prove very effective in filling this gap, if it developed materials suited to the Chinese cultural, economic, and educational context.

Comment author: geoffreymiller  (EA Profile) 16 August 2017 09:07:15PM 1 point [-]

From my perspective as an evolutionary psychologist, I wouldn't expect us to have reliable or coherent intuitions about utility aggregation for any groups larger than about 150 people, for any time-spans beyond two generations, or for any non-human sentient beings.

This is why consequentialist thought experiments like this so often strike me as demanding the impossible of human moral intuitions -- like expecting us to be able to reconcile our 'intuitive physics' concept of 'impetus' with current models of quantum gravity.

Whenever we take our moral intuitions beyond their 'environment of evolutionary adaptedness' (EEA), there's no reason to expect they can be reconciled with serious consequentialist analysis. And even within the EEA, there's no reason to expect out moral intuitions will be utilitarian rather than selfish + nepotistic + in-groupish + a bit of virtue-signaling.

View more: Prev