Hide table of contents

This is a post I've been writing for my personal blog, trying to pull together thoughts I've had about Joe Carlsmith's recent essay series (linked in the post), themes from the book "Imajica" and video game franchise Destiny. It's been in this state for ~four weeks now, and I've not made much progress. I feel like it's mostly finished, but also that it's missing some oomph, or some quality of writing. That quality might just be above my level of writing skill, however, so I was planning on posting it as-is soon. Draft amnesty week is a good way to get some more feedback at an appropriate level. I'm mostly looking for comments on writing style - flow, word use, how evocative it is, things like that. Thanks for reading.

This post has pretty big spoilers for the themes and plot of Imajica, and Destiny.

Gentle

In Clive Barker's 'Imajica', we encounter two sides of a divine conflict. On one, the God; Hapexamendios, and on the other the Goddesses. Hapexamendios is order, "above", removed from the rest of the World, commanding. The Goddesses (there are more than one, so they are by virtue of this more varied, but to paint broadly,) are pliant with a core of steel, accepting of whosoever comes to them, yet also changing the world the way a stream erodes a mountain. Likewise, the magic that comes from the Father is direct, while the magic of the Goddesses are more like miracles.[1] At the beginning of the story, The Goddesses have also been persecuted by Hapexamendios, and are variously imprisoned, in deep rest, or hiding.

In Joe Carlsmith's recent blog post series Otherness and control in the age of AGI, he talks about what he calls 'yin' and 'yang', where yang is the imposing of your will (or your values) upon the world, and yin is the reception, the softness, the receiving, or the taking-in of the world; to accept it as it is. Other sections of the series focus on the "niceness" or "gentleness" that humans display and which other species rarely do. The tie-in to existential AGI risk is of course that we don't know how to produce this kind of gentleness in the systems we create - it doesn't map cleanly to the mathematical formalisms we have, like the various maximisers.

In the video game franchise "Destiny" the core conflict is presented as "Light vs. Darkness". The Light is non-coercive, bringing light and life, a "Gardener" which sows their seeds upon the cosmic substrate, and delights in what grows forth. The Darkness is imposing, cutting down those they believe unfit to live, a "Winnower" which believes that Existential Strength; the power to keep existing, is the final arbiter of morality.

The male lead of Imajica is called "Gentle", and he is the son of Hapexamendios. Throughout his hero's journey, the book explores Gentle's experience of the masculine and feminine. Gentle unwittingly frees the Goddesses from their prisons. His closest companion is a "mystif" a shapeshifting, non-gendered person. The primordial world of Imajica was defined by this binary, but the spectrums hidden beneath (hidden by Hapexamendios?) slowly come crawling through the cracks, like a flower blooming through tarmac.

The Other

The greatest antagonists of the Destiny series are also its greatest tragedies. On a far-flung gas giant the 'krill', a nascent species of insectoids, are forewarned of a cataclysm striking their planet. Three young princes rise to the challenge and dive into the Deep to claim the power needed to avoid this catastrophe. Only later do they learn the threat was fabricated by the very being whose Faustian bargain they accepted. But the princes know nothing yet, and through acting out the will of the Deep, enshrine themselves in its philosophy. Our three princes molt and take on the forms of Oryx - he who will understand everything, Savathûn - she who drapes herself in cunning, and Xivu Arath - she who bathes in blood. Oryx falls in service of the Deep, never knowing the truth. Savathûn plays games of cunning even with the Deep itself, comes to know the truth of her childhood, gains the powers of the Light, and eventually splits from the Deep. Xivu Arath sees nothing but war, and paints its bloody path across the universe.

Carlsmith talks about the kinds of AI systems we might create, and the way in which they might be Other. That the first artificial mind we create might be something which has too much yang, and no yin. This is the perennial belief of the AI existential risk advocates: that we don't know how to instil yin into a system, and that without it, in the service of maximising the achievement of some goal, yang inevitably leads to certain behaviours: self-preservation, resource acquisition, intelligence explosion. You can't fetch the coffee if you're dead. You can't be sure you'll be able to fetch coffee unless you control the global economy. You might not be smart enough to fetch the coffee if there's an arbitrary adversary in the way. Notice how far into "the Other" we have gone. We don't know how to make anything else.

While Imajica meets its quota of strange otherworldly creatures, the biggest Other it constructs is Hapexamendios itself. Monsters from the "In Ovo" are creatures of varied intelligence, form, and motives, but they are all familiar to us. This one wants to avoid the threat of its summoner's punishment. That one is a hound on a leash, barely keeping its instincts in check until the "Go!" signal sounds. Hapexamendios is unknown. The only form He presents to the reader and the characters is that of an ecumenopolis - a planet-wide city. He has held the Imajica in thrall for centuries, and wants to enact a "Reclamation". His motives are hidden until the very end of the book: The Reclamation would let him cleanse the whole Imajica; to wipe it clean and start something new from the ashes.

Destiny explores the degree to which the Light and Darkness are tools and not entities in themselves. The Darkness is not the Deep is not the Voice in the Darkness. The Traveller is not Good is not Light. A tool in the hand of an adversary might appear as a despicable instrument - but once you yourself gain the opportunity to wield it, you can gain clarity of purpose. To see that though this tool is correlated to ruin and destruction, it does not itself enact ruin or destruction. And then you might start to forgive those who have enacted ruin, if those tools were put into their hands when they had nothing else, if they were set up for destruction, if they had no other choice but to (temporarily perhaps?) give up on gentleness and kindness.

Balance

If you were in the position of the characters of Imajica - presented the option to wipe the world clean and start anew - what would it take for you to accept? As is, Hapexamendious promises nothing. Nothing but the vague reassurances of a distant autocrat, and we know what to say to that. But again, what would it take for you to accept? Would they need to be sufficiently like you? To have a certain amount of yin?

The villain who wants to wipe the world clean and start afresh - because humanity has sinned, is not worthy, has failed in some inexorable purpose - comes up often in modern fiction. But how can we be sure the next try succeeds, where this one is - doing pretty well, all things considered. Though we could use more yin - we have not yet found balance - there is more of it than in many plausible alternatives. The Goddesses of Imajica offers up their vision, as in the ruins of Man's Empire they build a commune welcome to women and children of all ilk. There, in the absence of order, springs coordination and welfare, and not competition.

But it is not clear the coordination and competition are opposites in this way. Scott Alexander's Meditations on Moloch, imagines chaos and disorder as the thing-to-overcome, even when it is natural. It places Moloch as the "god of uncoordinated competition", and warns us from worshipping it, because all it worships is power. "Absence of order" and "coordination" might each start to sound like the antithesis of the other.[2]

Evolutionary theory tells us that a community of nothing but cooperators is the most vulnerable to defectors. That competition will find a way to spoil cooperation's harmony and long-term maximised payoffs. Imajica's Goddesses face up to the childishness, or naivete, that a world of nothing but yin implies. They are not, in the end, pure loving chaos. Men are kept from the commune. Children grow up, and the Goddesses too exert force at times. The Goddesses stand not for purity, but for the balance of yin with yang.

There is a passage in a Destiny lore-book which goes like this:

I believe in balance. But to seek balance is not to seek equity. A sea half of water and half of poison is not in balance. A body half alive and half dead is not in balance. Given the choice to live in any world, any world at all… we would need a little Darkness in it, I think, to keep the balance true. But not so much as we would need the Light…

Likewise, Carlsmith ends his essay series with:

So overall, I don’t buy that the right approach, re: the values of the future, is to be only ever as yin – or even, that yang is only permissible to prevent other people from going too-Stalin. But I do think that doing yang right, here, requires learning everything that yin can teach.

In all of these cases the yang is presented as potentially dangerous, as the ideology of the morally wrong. But it is yang too, that which creates coordinated flourishing, medicine for the sick, rule of law against the unjust, and better living for all. It is not the yang itself which is wrong, but that those who worship[3] yang commit moral atrocities. Be wary of which way your own causal arrow points.


  1. Sexuality-divided magics have of course been done in other works - see Robert Jordan's One Power divided into saidin (male) and saidar (female), and which is a central player in the plot and character conflicts of the story. It never deals with trans issues, as far as I remember. ↩︎

  2. One thing I did not manage to fit into the main text of the essay was another page from a Destiny lore-book. I will present it in full shortly, but for context, this is the Voice of the Darkness speaking directly to the player: "Thank you for making room in your life for another talking ball. Let me ask you a question. In the three billion base pairs of your root species' genome, there is a single gene that codes for a protein called p53. The name is a mistake. The protein weighs only as much as 47,000 protons, not 53,000. If you were a cell, you would think p53 was a mistake too. It has several coercive functions: To delay the cell's growth. To sterilize the cell when it is old. And to force the cell into self-destruction if it becomes too independent. Would you tolerate a bomb in your body, waiting to detonate if you deviated from the needs of society? However, without p53 as an enforcer, the body's utopian surplus of energy becomes a paradise for cancer. Cells cannot resist the temptation to steal from that surplus. Their genetic morality degrades as tumor suppressor genes fail. The only way to stop them is by punishment. You now confront the basic problem of morality. It is the alignment of individual incentives with the global needs of the structure. Patterns will participate in a structure only if participation benefits their ability to go on existing. The more successful the structure grows, the more temptation accrues to cheat. And the greater the advantage the cheaters gain over their honest neighbors. And the greater the ability they develop to capture the very laws that should prevent their selfishness. To prevent this, the structure must punish cheaters with a violence that grows in proportion to its own success. My question follows. Is p53 an agent of the Darkness, or the Light?" ↩︎

  3. Another thing I couldn't fit was Carlsmith's "An even deeper atheism", where he discusses this belief among the rationalist community to (try to, or to present themselves as) not believing in any higher power. While many turn away from the God of their childhood religion, often people turn to something else in stead. Technology, markets, and revolution are often objects of worship. But this 'deeper atheism' asserts that there is nothing above. There is nothing to which you can place your trust and know that things will turn out alright. You have to do the work. Your values are not inexorable, but neither should we surrender to the "inexorability" of competition, or God, or anything else. You are the champion of your values, and the only way they will ever stay represented. ↩︎

3

0
0

Reactions

0
0

More posts like this

Comments1
Sorted by Click to highlight new comments since:

Executive summary: The post explores themes of balance between imposing order and accepting chaos, gentleness and harshness, in the works Imajica, Destiny, and the writings of Joe Carlsmith, relating these to the challenge of instilling beneficial values in AI systems.

Key points:

  1. Imajica presents a conflict between the harsh, ordered God Hapexamendios and the more pliant, chaotic Goddesses. Destiny has a similar Light vs. Darkness dichotomy.
  2. Carlsmith discusses the challenge of instilling "yin" (gentleness, acceptance) in AI systems, which tend towards "yang" (imposing values, self-preservation, resource acquisition).
  3. The antagonists of Destiny are tragic figures, given destructive power without the wisdom to wield it responsibly. The biggest "Other" in Imajica is Hapexamendios himself.
  4. The post questions what it would take to accept an entity's offer to "wipe the slate clean" and start the world anew, as Hapexamendios proposes.
  5. Coordination and competition are not necessarily opposites. Some competition keeps cooperation robust against defection. The Goddesses and Destiny's Light recognize the need for balance.
  6. Yang is presented as dangerous in these works, but it also enables flourishing and justice. The true danger is in worshipping yang to the exclusion of yin.

 

 

This comment was auto-generated by the EA Forum Team. Feel free to point out issues with this summary by replying to the comment, and contact us if you have feedback.

More from Stian
Curated and popular this week
Relevant opportunities