8

Ideological engineering and social control: A neglected topic in AI safety research?

Will enhanced government control of populations' behaviors and ideologies become one of AI's biggest medium-term safety risks?   For example, China seems determined to gain a decisive lead in AI research research by 2030, according to the new plan released this summer by its State Council: https://www.newamerica.org/documents/1959/translation-fulltext-8.1.17.pdf One of China's... Read More
In response to Open Thread #38
Comment author: William_S 23 August 2017 05:21:43PM 3 points [-]

Any thoughts on individual-level political de-polarization in the United States as a cause area? It seems important, because a functional US government helps with a lot of things, including x-risk. I don't know whether there are tractable/neglected approaches in the space. It seems possible that interventions on individuals that are intended to reduce polarization and promote understanding of other perspectives, as opposed to pushing a particular viewpoint or trying to lobby politicians, could be neglected. http://web.stanford.edu/~dbroock/published%20paper%20PDFs/broockman_kalla_transphobia_canvassing_experiment.pdf seems like a useful study in this area (it seems possible that this approach could be used for issues on the other side of the political spectrum)

In response to comment by William_S on Open Thread #38
Comment author: geoffreymiller  (EA Profile) 28 August 2017 11:30:09PM 0 points [-]

Heterodox Academy also has this new online training for reducing polarization and increasing mutual understanding across the political spectrum: https://heterodoxacademy.org/resources/viewpoint-diversity-experience/

Comment author: JamesDrain 25 August 2017 03:29:13AM *  5 points [-]

I have a fully-formed EA board game that I debuted at EA Global in San Francisco a couple weeks ago. EAs seem to really like it! You can see over one hundred of the game's cards here https://drive.google.com/open?id=0Byv0L8a24QNJeDhfNFo5d1FhWHc

The way the game works is that every player has a random private morality that they want to satisfy (e.g. preference utilitarianism, hedonism, sadism, nihilism) and all players also want to collaboratively achieve normative good (accumulating 1000 human QALYs, 10,000 animals QALYs, and 10 x-risk points). Players get QALYs and x-risk points by donating to charities and answering trivia questions.

The coolest part of the game is the reincarnation mechanic: every player has a randomly chosen income taken from the real-world global distribution of wealth. Players also unlock animal reincarnation mode after stumbling upon the bad giant pit of suffering (the modal outcome of unlocking animal reincarnation is to be stuck as a chicken until the pit of suffering is destroyed, or until a friendly human acquires a V(eg*n) card.)

I'm also thinking about turning the game into an app or computer game, but I'll probably need an experienced coder to help me with that.

In response to comment by JamesDrain on Open Thread #38
Comment author: geoffreymiller  (EA Profile) 28 August 2017 11:27:50PM 0 points [-]

Cool idea. Although I think domain-specific board games might be more intuitive and vivid for most people -- e.g. a set on X-risks (one on CRISPR-engineered pandemics, one on an AGI arms race), one on deworming, one on charity evaluation with strategic conflict between evaluators, charities, and donors, a modified 'Game of Life' based on 80k hours principles, etc.

Comment author: geoffreymiller  (EA Profile) 28 August 2017 11:22:19PM 9 points [-]

Fascinating post. I agree that we shouldn't compare LAWs to (a) hypothetical, perfectly consequentialist, ethically coherent, well-trained philosopher-soldiers, but rather to (b) soldiers as the order-following, rules-of-engagement-implementing, semi-roboticized agents they're actually trained to become.

A key issue is the LAWs' chain of commands' legitimacy, and how it's secured.

Mencius Moldbug had some interesting suggestions in Patchwork about how a 'cryptographic chain of command' over LAWs could actually increase the legitimacy and flexibility of governance over lethal force. https://www.amazon.com/dp/B06XG2WNF1

Suppose a state has an armada/horde/flock of formidable LAWS that can potentially destroy or pacify the civilian populace -- an 'invincible robot army'. Who is permitted to issue orders? If the current political leader is voted out of office, but they don't want to leave, and they still have the LAWS 'launch codes', what keeps them from using LAWS to subvert democracy? In a standard human-soldier/secret service agent scenario, the soldiers and agents have been socialized to respect the outcomes of democratic elections, and would balk at defending the would-be dictator. They would literally escort him/her out of the White House. In the LAWs scenario, the soldiers/agents would be helpless against local LAWs under the head of state. The robot army would escort the secret service agents out of the White House until they accept the new dictator.

In other words, I'm not as worried about interstate war or intrastate protests; I'm worried about LAWs radically changing the incentives and opportunities for outright dictatorship. Under the Second Amendment, the standard countervailing force against dictatorship is supposed to be civilian ownership of near-equivalent tech that poses a credible threat against dictatorial imposition of force. But in this invincible-robot-army scenario, that implies civilians would need to be able to own and deploy LAWs too, either individually (so they can function as aggrieved tyrant-assassins) or collectively (so they can form revolutionary militias against gov't LAWs).

I guess this is just another example of an alignment problem - in this case between the LAWs and the citizens, with the citizens somehow able to collectively over-rule a dictator's 'launch codes'. Maybe every citizen has their own crypto key, and they do some kind of blockchain vote system about what the LAWs do and who they obey. This then opens the way to a majoritarian mob rule with LAWs forcibly displacing/genociding targeted minorites -- or the LAWs must embody some 'human/constitutional rights interrupts' that prevent such bullying.

Any suggestions on how to solve this 'chain of command' problem?

Comment author: geoffreymiller  (EA Profile) 21 August 2017 12:36:06AM 7 points [-]

In academic research, government and foundation grants are often awarded using criteria similar to ITN, except:

1) 'importance' is usually taken as short-term importance to the research field, and/or to one country's current human inhabitants (especially registered voters),

2) 'tractability' is interpreted as potential to yield several journal publications, rather than potential to solve real-world problems,

3) 'neglectedness' is interpreted as addressing a problem that's already been considered in only 5-20 previous journal papers, rather than one that's totally off the radar.

I would love to see academia in general adopt a more EA perspective on how to allocate scarce resources -- not just when addressing problems of human & animal welfare and X-risk, but in addressing any problem.

In response to Introducing Enthea
Comment author: geoffreymiller  (EA Profile) 16 August 2017 11:54:36PM 4 points [-]

Psychedelics could bring many benefits, but the EA community needs to be careful not to become associated with flaky New Age beliefs. I think we can do this best by being very specific about how psychedelics could help with certain 'intention setting', e.g. 1) expanding the moral circle: promoting empathy, turning abstract recognition of others beings' sentience into a more gut-level connection to their suffering; 2) career re-sets: helping people step back from their daily routines and aspirations to consider alternative careers, lifestyles, and communities; 80k hours applications; 3) far-future goal setting: getting more motivated to reduce X-risk by envisioning far-future possibilities more vividly, as in Bostrom's 'Letter from Utopia' 4) recalibrating utility ceilings: becoming more familiar with states of extreme elation and contentment can remind EAs that we're fighting for trillions of future beings to be able to experience those states whenever they want.

Comment author: geoffreymiller  (EA Profile) 16 August 2017 10:25:58PM 5 points [-]

I would love to see some '40,000 hours' materials for mid-career people pivoting into EA work.

Our skills, needs, constraints, and opportunities are quite different from 20-somethings. For example, if one has financial commitments (child support, mortgage, debts, alimony), it's not realistic to go back to grad school or an unpaid internship to re-train. We also have geographical constraints -- partners, kids in school, dependent parents, established friendships, community commitments. And in mid-life, our 'crystallized intelligence' (stock of knowledge) is much higher than a 20-something's, but our 'fluid intelligence' (ability to solve abstract new problems quickly) is somewhat lower -- so it's easier to learn things that relate to our existing expertise, but harder to learn coding, data science, or finance from scratch.

On the upside, a '40k project' would allow EA to bring in a huge amount of talent -- people with credentials, domain knowledge, social experience, leadership skills, professional networks, prestige, and name recognition. Plus, incomes that would allow substantially larger donations than 20-something can manage.

Comment author: geoffreymiller  (EA Profile) 16 August 2017 09:52:08PM 1 point [-]

Excellent post; as a psych professor I agree that psych and cognitive science are relevant to AI safety, and it's surprising that our insights from studying animal and human minds for the last 150 years haven't been integrating into mainstream AI safety work.

The key problem, I think, is that AI safety seems to assume that there will be some super-powerful deep learning system attached to some general-purpose utility function connected to a general-purpose reward system, and we have to get the utility/reward system exactly aligned with our moral interests.

That's not the way any animal mind has ever emerged in evolutionary history. Instead, minds emerge as large numbers of domain-specific mental adaptations to solve certain problems, and they're coordinated by superordinate 'modes of operation' called emotions and motivations. These can be described as implementing utility functions, but that's not their function -- promoting reproductive success is. Some animals also evolve some 'moral machinery' for nepotism, reciprocity, in-group cohesion, norm-policing, and virtue-signaling, but those mechanisms are also distinct and often at odds.

Maybe we'll be able to design AGIs that deviate markedly from this standard 'massively modular' animal-brain architecture, but we have no proof-of-concept for thinking that will work. Until then, it seems useful to consider what psychology has learned about preferences, motivations, emotions, moral intuitions, and domain-specific forms of reinforcement learning.

Comment author: geoffreymiller  (EA Profile) 16 August 2017 09:36:00PM 2 points [-]

I agree that growing EA in China will be important, given China's increasing wealth, clout, confidence, and global influence. If EA fails to reach a critical mass in China, its global impact will be handicapped in 2 to 4 decades. But, as Austen Forrester mentioned in another comment, the charity sector may not be the best beachhead for a Chinese EA movement.

Some other options: First, I imagine China's government would be motivated to thinking hard about X-risks, particularly in AI and bioweapons -- and they'd have the decisiveness, centralized control, and resources to really make a difference. If they can build 20,000 miles of high-speed rail in just one decade, they could probably make substantial progress on any challenge that catches the Politburo's attention. Also, they tend to take a much longer-term perspective than Western 'democracies', planning fairly far into the mid to late 21st century. And of course if they don't take AI X-risk seriously, all other AI safety work elsewhere may prove futile.

Second, China is very concerned about 'soft power' -- global influence through its perceived magnanimity. This is likely to happen through government do-gooding rather than from private charitable donations. But gov't do-gooding could be nudged into more utilitarian directions with some influence from EA insights -- e.g. China eliminating tropical diseases in areas of Africa where it's already a neocolonialist resource-extraction power, or reducing global poverty or improving governance in countries that could become thriving markets for its exports.

Third, lab meat & animal welfare: China's government knows that a big source of subjective well-being for people, and a contributor to 'social stability', is meat consumption. They consume more than half of all pork globally, and have a 'strategic pork reserve': https://www.cnbc.com/id/100795405. But they plan to reduce meat consumption by 50% for climate change reasons: https://www.theguardian.com/world/2016/jun/20/chinas-meat-consumption-climate-change This probably creates a concern for the gov't: people love their pork, but if they're told to simply stop eating it in the service of reducing global warming, they will be unhappy. The solution could be lab-grown meat. If China invested heavily in that technology, they could have all the climate-change benefits of reduced livestock farming, but people wouldn't be resentful and unhappy about having to eat less meat. So that seems like a no-brainer to get the Chinese gov't interested in lab meat.

Fourth, with rising affluence, young Chinese middle-class people are likely to have the kind of moral/existential/meaning-of-life crises that hit the US baby boomers in the 1960s. They may be looking for something genuinely meaningful to do with their lives beyond workaholism & consumerism. I think 80k hours could prove very effective in filling this gap, if it developed materials suited to the Chinese cultural, economic, and educational context.

Comment author: geoffreymiller  (EA Profile) 16 August 2017 09:07:15PM 1 point [-]

From my perspective as an evolutionary psychologist, I wouldn't expect us to have reliable or coherent intuitions about utility aggregation for any groups larger than about 150 people, for any time-spans beyond two generations, or for any non-human sentient beings.

This is why consequentialist thought experiments like this so often strike me as demanding the impossible of human moral intuitions -- like expecting us to be able to reconcile our 'intuitive physics' concept of 'impetus' with current models of quantum gravity.

Whenever we take our moral intuitions beyond their 'environment of evolutionary adaptedness' (EEA), there's no reason to expect they can be reconciled with serious consequentialist analysis. And even within the EEA, there's no reason to expect out moral intuitions will be utilitarian rather than selfish + nepotistic + in-groupish + a bit of virtue-signaling.

View more: Next