In response to Utopia In The Fog
Comment author: MikeJohnson 31 March 2017 09:13:14PM 3 points [-]

There's a lot of value in having an AI safety orthodoxy for coordination purposes; there's also a lot of value in this sort of heterodox criticism of the orthodoxy. Thanks for posting.

One additional area of orthodoxy that I think could use more critique is the community's views on consciousness. A few thoughts here (+comments): http://effective-altruism.com/ea/14t/principia_qualia_blueprint_for_a_new_cause_area/

Also: nobody seems to be really looking into the state of AI safety & x-risk memes inside of China. Whether they're developing a different 'availability cascade' seems hugely important and under-studied.

Comment author: Brian_Tomasik 19 January 2017 06:19:09PM 2 points [-]

we could use odd definitions of suffering to conclude that animals probably don’t feel pain, or that current chatbots can feel pain, or that the suffering which happens when a cis white man steps on a nail, is less than the suffering which happens when a bisexual black female steps on a nail, or vice-versa.

But doing so would amount to shifting the goalpost, which is a way of cheating at arguments whether there's a single definition of a word or not. :)

It's similar to arguments over abortion of very early embryos. One side calls a small clump of cells "a human life", and the other side doesn't. There's no correct answer; it just depends what you mean by that phrase. But the disagreement isn't rendered trivial by the lack of objectivity of a single definition.

will almost certainly be co-opted by future intellectual fashions

If by this you mean society's prevailing concepts and values, then yes. But everything is at the mercy of those. If reducing your precisely defined version of suffering falls out of fashion, it won't matter that it has a crisp definition. :)

some future Effective Altruism movement may very well work to maximize what we today would call suffering.

Hm, that doesn't seem too likely to me (more likely is that society becomes indifferent to suffering), except if you mean that altruists might, e.g., try to maximize the amount of sentience that exists, which would as a byproduct entail creating tons of suffering (but that statement already describes many EAs right now).

My challenge to you is to find a way out of this ‘repugnant conclusion’ also. Or to disprove that I’ve found a way out of it, of course. :)

I think your solution, even if true, doesn't necessarily help with goal drift / Moloch stuff because people still have to care about the kind of suffering you're talking about. It's similar to moral realism: even if you find the actual moral truth, you need to get people to care about it, and most people won't (especially not future beings subject to Darwinian pressures).

Comment author: MikeJohnson 20 January 2017 09:11:24PM *  2 points [-]

[constructivism implies that] society’s definition of suffering, and any institutions we build whose mission is to reduce suffering, will almost certainly be co-opted by future intellectual fashions

If by this you mean society's prevailing concepts and values, then yes. But everything is at the mercy of those. If reducing your precisely defined version of suffering falls out of fashion, it won't matter that it has a crisp definition. :)

Our definition of electricity may evolve over time, in accordance with new developments in the foundational physics, but we’re unlikely to chuck quantum field theory in favor of some idiosyncratic theory of crystal chakras. If we discover the universe’s equation for valence, we’re unlikely to find our definition of suffering at the mercy of intellectual fads.

And so, I fear that if we’re constructivists about suffering, then we should expect a very dark scenario: that society’s definition of suffering, and any institutions we build whose mission is to reduce suffering, will almost certainly be co-opted by future intellectual fashions. And, in fact, that given enough time and enough Moloch, society’s definition of suffering could in fact invert, and some future Effective Altruism movement may very well work to maximize what we today would call suffering.

Hm, that doesn't seem too likely to me (more likely is that society becomes indifferent to suffering), except if you mean that altruists might, e.g., try to maximize the amount of sentience that exists, which would as a byproduct entail creating tons of suffering (but that statement already describes many EAs right now).

I agree that this seems unlikely, but it seems like you grant that such a values—inversion is possible, and say that it wouldn’t be a bad thing, because there’s no fundamental moral truth (moral nihilism). But I think that, unambiguously, cats being lit on fire is an objectively bad thing. Even if time and Moloch happen to twist the definition of ‘suffering’ such that future utilitarian EAs want to tile the universe with burning cats, I completely reject that such an intellectual fashion could be right.

I think most people would strongly agree with this moral realist position, rather than the moral nihilist position- that this specific thing is actually and unambiguously is bad, and that any definition of suffering that wouldn’t say it’s bad is wrong.

I think your solution, even if true, doesn't necessarily help with goal drift / Moloch stuff because people still have to care about the kind of suffering you're talking about. It's similar to moral realism: even if you find the actual moral truth, you need to get people to care about it, and most people won't (especially not future beings subject to Darwinian pressures).

Yeah, I mostly agree with this— Andres covers some of this with this post. I feel great urgency to figure this out while we’re still in the non-malthusian time Robin Hanson calls the dreamtime. If we don’t figure out what has value and slide into a highly Darwinian/Malthusian/Molochian context, then I fear that could be the end of value.

Comment author: Brian_Tomasik 20 January 2017 10:48:11AM *  2 points [-]

So I would say very strongly that we can’t both say that electricity is subjective and everyone can have their own arbitrary poetic definition of what it is and how it works, but also do interesting and useful things with it.

As long as we're both using the same equations of physics to describe the phenomenon, it seems that exactly how we define "electricity" may not matter too much. The most popular interpretation of quantum mechanics is "shut up and calculate".

As another analogy, "life" has a fuzzy, arbitrary boundary, but that doesn't prevent us from doing biology.

If someone makes a strong assertion that something is bad and that we should work to reduce its prevalence, then they’re also implying it’s real in a non-trivial sense; if something is not real, then it cannot be bad in an actionable sense.

An example I like to use is "justice". It's clear to many people that injustice is bad, even though there's no crisp, physics-based definition of injustice.

“Well, blegblarg doesn’t have a crisp definition, it’s more of a you-know-it-when-you-see-it thing where there’s no ‘correct’ definition of blegblarg and we can each use our own moral compass to determine if something is blegblarg, but there’s definitely a lot of it out there and it’s clearly bad so we should definitely work to reduce it!”

Replace "blegblarg" with "obscenity", and you have an argument that many people suffering from religious viruses would endorse.

But you’re clearly not a moral nihilist

I am. :) At least by this definition: "Moral nihilists consider morality to be constructed, a complex set of rules and recommendations that may give a psychological, social, or economical advantage to its adherents, but is otherwise without universal or even relative truth in any sense."

my apologies if anything I've said comes across as rude

I was worried about the same in reverse. I didn't find your comments rude. :)

Comment author: MikeJohnson 20 January 2017 08:44:18PM 3 points [-]

I was worried about the same in reverse. I didn't find your comments rude. :)

Good! I’ll charge forward then. :)

As long as we're both using the same equations of physics to describe the phenomenon, it seems that exactly how we define "electricity" may not matter too much. The most popular interpretation of quantum mechanics is "shut up and calculate".

That is my favorite QM interpretation! But following this analogy, I’m offering a potential equation for electricity, but you’re saying that electricity doesn’t have an equation because it’s not ‘real’, so it doesn’t seem like you will ever be in a position to calculate.

“Well, blegblarg doesn’t have a crisp definition, it’s more of a you-know-it-when-you-see-it thing where there’s no ‘correct’ definition of blegblarg and we can each use our own moral compass to determine if something is blegblarg, but there’s definitely a lot of it out there and it’s clearly bad so we should definitely work to reduce it!”

Replace "blegblarg" with "obscenity", and you have an argument that many people suffering from religious viruses would endorse.

But that doesn’t address the concern: if you argue that something is bad and we should work to reduce it, but also say there’s no correct definition for it and no wrong definition for it, what are you really saying? You note elsewhere that “We can interpret any piece of matter as being conscious if we want to,” and imply something similar about suffering. I would say that a definition that allows for literally anything is not a definition; an ethics that says something is bad, but notes that it’s impossible to ever tell whether any particular thing is bad, is not an ethics.

An example I like to use is "justice". It's clear to many people that injustice is bad, even though there's no crisp, physics-based definition of injustice.

This doesn’t seem to match how you use the term ‘suffering’ in practice. E.g., we could claim that “protons oppress electrons” or “there’s injustice in fundamental physics” — but this is obviously nonsense, and from a Wittgensteinian “language game” point of view, what's happening is that we’re using perfectly good words in contexts where they break down. But you do want to say that there could be suffering in fundamental physics, and potentially the far future. It looks like you want to have your cake and eat it too, and say that (1) “suffering” is a fuzzy linguistic construct, like “injustice” is, but also that (2) we can apply this linguistic construct of “suffering” to arbitrary contexts without it losing meaning. This seems deeply inconsistent.

But you’re clearly not a moral nihilist

I am. :) At least by this definition: "Moral nihilists consider morality to be constructed, a complex set of rules and recommendations that may give a psychological, social, or economical advantage to its adherents, but is otherwise without universal or even relative truth in any sense."

That definition doesn’t seem to leave much room for ethical behavior (or foundational research!), merely selfish action. This ties into my notion above, that you seem to have one set of stated positions (extreme skepticism & constructivism about qualia & suffering, moral nihilism for the purpose of ‘psychological, social, or economical advantage’), but show different revealed preferences (which seem more altruistic, and seem to assume something close to moral realism).

The challenge in this space of consciousness/valence/suffering research is to be skeptical-yet-generative: to spot and explain the flaws in existing theories, yet also to constantly search for and/or build new theories which have the potential to avoid these flaws.

You have many amazing posts doing the former (I particularly enjoyed this piece ) but you seem to have given up on the latter, and at least in these replies, seem comfortable with extreme constructivism and moral nihilism. However, you also seem to implicitly lean on valence realism to avoid biting the bullet on full-out moral nihilism & constructivism— your revealed preferences seem to be that you still want meaning, you want to say suffering is actually bad, I assume you don’t think it’s 100% arbitrary whether we say something is suffering or not. But these things are not open to a full-blown moral nihilist.

Anyway, perhaps you would have very different interpretations on these things. I would expect so. :) I'm probing your argument to see what you do think. But in general, I agree with the sentiments of Scott Aaronson:

Yes, it’s possible that things like the hard problem of consciousness, or the measurement problem in quantum mechanics, will never have a satisfactory resolution. But even if so, building a complicated verbal edifice whose sole purpose is to tell people not even to look for a solution, to be satisfied with two “non-overlapping magisteria” and a lack of any explanation for how to reconcile them, never struck me as a substantive contribution to knowledge. It wasn’t when Niels Bohr did it, and it’s not when someone today does it either.

I want a future where we can tell each other to “shut up and calculate”. You may not like my solution for grounding what valence is (though I’m assuming you haven’t read Principia Qualia yet), but I hope you don’t stop looking for a solution.

Comment author: Brian_Tomasik 19 January 2017 06:19:09PM 2 points [-]

we could use odd definitions of suffering to conclude that animals probably don’t feel pain, or that current chatbots can feel pain, or that the suffering which happens when a cis white man steps on a nail, is less than the suffering which happens when a bisexual black female steps on a nail, or vice-versa.

But doing so would amount to shifting the goalpost, which is a way of cheating at arguments whether there's a single definition of a word or not. :)

It's similar to arguments over abortion of very early embryos. One side calls a small clump of cells "a human life", and the other side doesn't. There's no correct answer; it just depends what you mean by that phrase. But the disagreement isn't rendered trivial by the lack of objectivity of a single definition.

will almost certainly be co-opted by future intellectual fashions

If by this you mean society's prevailing concepts and values, then yes. But everything is at the mercy of those. If reducing your precisely defined version of suffering falls out of fashion, it won't matter that it has a crisp definition. :)

some future Effective Altruism movement may very well work to maximize what we today would call suffering.

Hm, that doesn't seem too likely to me (more likely is that society becomes indifferent to suffering), except if you mean that altruists might, e.g., try to maximize the amount of sentience that exists, which would as a byproduct entail creating tons of suffering (but that statement already describes many EAs right now).

My challenge to you is to find a way out of this ‘repugnant conclusion’ also. Or to disprove that I’ve found a way out of it, of course. :)

I think your solution, even if true, doesn't necessarily help with goal drift / Moloch stuff because people still have to care about the kind of suffering you're talking about. It's similar to moral realism: even if you find the actual moral truth, you need to get people to care about it, and most people won't (especially not future beings subject to Darwinian pressures).

Comment author: MikeJohnson 20 January 2017 03:01:53AM *  3 points [-]

Thanks for the thoughts! Here's my attempt at laying out a strong form of why I don't think constructivism as applied to ethics & suffering leads to productive areas:

Imagine someone arguing that electromagnetism was purely a matter of definitions- there’s no “correct” definition of electricity, so how one approaches the topic and which definition one uses is ultimately a subjective choice.

But now imagine they also want to build a transistor. Transistors are, in fact, possible, and so it turns out that there is a good definition of electricity, by way of quantum theory, and of course many bad ones that don’t ‘carve reality at the joints’.

So I would say very strongly that we can’t both say that electricity is subjective and everyone can have their own arbitrary poetic definition of what it is and how it works, but also do interesting and useful things with it.

Likewise, my claim is that we can be a subjectivist about qualia and about suffering and say that how we define them is rather arbitrary and ultimately subjective, or we can say that some qualia are better than others and we should work to promote more good qualia and less bad qualia. But I don’t think we can do both at the same time. If someone makes a strong assertion that something is bad and that we should work to reduce its prevalence, then they’re also implying it’s real in a non-trivial sense; if something is not real, then it cannot be bad in an actionable sense.

Imagine that tomorrow I write a strong denouncement of blegblarg on the EA forum. I state that blegblarg is a scourge upon the universe, and we should work to rid ourselves of it, and all right-thinking people should agree with me. People ask me, “Mike…. I thought your post was interesting, but…. what the heck is blegblarg??” - I respond that “Well, blegblarg doesn’t have a crisp definition, it’s more of a you-know-it-when-you-see-it thing where there’s no ‘correct’ definition of blegblarg and we can each use our own moral compass to determine if something is blegblarg, but there’s definitely a lot of it out there and it’s clearly bad so we should definitely work to reduce it!”

This story would have no happy ending. Blegblarg can’t be a good rallying cry, because I can’t explain what it is. I can’t say it’s good or bad in a specific actionable sense, for the same reason. One person’s blegblarg is another person’s blargbleg, you know? :)

I see a strict reading of the constructivist project as essentially claiming similar things about suffering, and ultimately leading to concluding that what is, and isn't, suffering, is fundamentally arbitrary-- i.e., it leads to post-modern moral nihilism. But you’re clearly not a moral nihilist, and FRI certainly doesn’t see itself as nihilist. In my admittedly biased view of the situation, I see you & FRI circling around moral realism without admitting it. :) Now, perhaps my flavor of moral realism isn’t to your liking- perhaps you might come to a completely different principled conclusion about what qualia & valence are. But I do hope you keep looking.

p.s. I tend to be very direct when speaking about these topics, and my apologies if anything I've said comes across as rude. I think we differ in an interesting way and there may be updates in this for both of us.

Comment author: Brian_Tomasik 19 January 2017 12:59:59PM 1 point [-]

Hm, I would think that hedonic adaptation/habituation could be applied to stimuli from pleasure-ceptors fairly easily?

Comment author: MikeJohnson 19 January 2017 05:24:28PM 2 points [-]

Hmm-- I'd suggest that if pleasure-ceptors are easy contextually habituated, they might not be pleasure-ceptors per se.

(Pleasure is easily habituated; pain is not. This is unfortunate but seems adaptive, at least in the AE...)

My intuition is that if an organism did have dedicated pleasure-ceptors, it would probably immediately become its biggest failure-point (internal dynamics breaking down) and attack surface (target for others to exploit in order to manipulate behavior, which wouldn't trigger fight/flight like most manipulations do).

Arguably, we do see both of these things happen to some degree with regard to "pseudo-pleasure-ceptors" in the pelvis(?).

Comment author: Brian_Tomasik 17 January 2017 10:03:56AM 3 points [-]

One possible explanation why we have nociceptors but not direct pleasure-ceptors is that there's no stimulus that's always fitness-enhancing (or is there?), while flames, skin wounds, etc. are always bad. Sugar receptors usually convey pleasure, but not if you're full, nauseous, etc.

Also, we can't have simple pleasure-ceptors for beautiful images or music because those stimuli require complex processing by visual or auditory cortices; there's no "pleasant music molecule" that can stimulate a pleasure-ceptor neuron the way there are pleasant-tasting gustatory molecules.

Comment author: MikeJohnson 19 January 2017 07:24:34AM 2 points [-]

there's no stimulus that's always fitness-enhancing (or is there?), while flames, skin wounds, etc. are always bad. Sugar receptors usually convey pleasure, but not if you're full, nauseous, etc.

Yeah, strongly agree.

Additionally, accidentally wireheading oneself had to have been at least a big potential problem during evolution, which would strongly select against anything like a pleasure-ceptor.

Comment author: Brian_Tomasik 17 January 2017 09:46:19AM *  2 points [-]

Anything piped through the complexity of the brain will look complex, regardless of how simple or complex it starts out as.

Fair enough. :) By analogy, even if pesticide regulation looks complex, the molecular structure of a single insecticide molecule is more crisp.

there doesn't seem to be an equivalent document describing what suffering research is if we assume that consciousness should be thought of more as a linguistic confusion than a 'real' thing

Various of my essays mention examples of my intuitions on the topic, and this piece discusses one framework for thinking about the matter. But I envision this project as more like interpreting the themes and imagery of Shakespeare than like a comprehensive scientific program. It's subjective, personal, and dependent on one's emotional whims. Of course, one can choose to make it more formalized if one prefers, like formalized preference utilitarianism does.

Comment author: MikeJohnson 18 January 2017 11:44:19PM 2 points [-]

Our question is: For a given physical system, what kinds of emotion(s) is it experiencing and how good/bad are they? The answers will not be factual in any deep ontological sense, since emotions and moral valence are properties that we attribute to physics. Rather, we want an ethical theory of how to make these judgments.

Certainly, Barrett makes a strong case that statements about emotions “will not be factual in any deep ontological sense,” because they aren’t natural kinds. My argument is that valence probably is a natural kind, however, and so we can make statements about it that are as factual as statements about the weak nuclear force, if (and only if) we find the right level of abstraction by which to view it.

When I began discussing utilitarianism in late 2005, a common critique from friends was: "But how can you measure utility?" Initially I replied that utility was a real quantity, and we just had to do the best we could to guess what values it took in various organisms. Over time, I think I grew to believe that while consciousness was metaphysically real, the process of condensing conscious experiences into a single utility number was an artificial attribution by the person making the judgment. In 2007, when a friend pressed me on how I determined the net utility of a mind, I said: "Ultimately I make stuff up that seems plausible to me." In late 2009, I finally understood that even consciousness wasn't ontologically fundamental, and I adopted a stance somewhat similar to, though less detailed than, that of the present essay.

I would say I’ve undergone the reverse process. :)

Your implication is that questions of consciousness & suffering are relegated to ‘spiritual poetry’ and can only be ‘debated in the moral realm’ (as stated in some of your posts). But I would suggest this is rather euphemistic, and runs into failure modes that are worrying.

The core implication seems to be that there are no crisp facts of the matter about what suffering is, or which definition is the ‘correct’ one, and so it's ultimately a subjective choice which definition we use. This leads to insane conclusions: we could use odd definitions of suffering to conclude that animals probably don’t feel pain, or that current chatbots can feel pain, or that the suffering which happens when a cis white man steps on a nail, is less than the suffering which happens when a bisexual black female steps on a nail, or vice-versa. I find it very likely that there are people making all these claims today.

Now, I suspect you and I have similar intuitions about these things: we both think animals can feel pain, whereas current chatbots probably can’t, and that race almost certainly doesn’t matter with respect to capacity to suffer. I believe I can support these intuitions from a principled position (as laid out in Principia Qualia). But being a functionalist, and especially if our moral intuitions and definitions of suffering are “subjective, personal, and dependent on one’s emotional whims,” then it would seem that your support of these intuitions is in some sense arbitrary— they are your spiritual poetry, but other people can create different spiritual poetry that comes from very different directions.

And so, I fear that if we’re constructivists about suffering, then we should expect a very dark scenario: that society’s definition of suffering, and any institutions we build whose mission is to reduce suffering, will almost certainly be co-opted by future intellectual fashions. And, in fact, that given enough time and enough Moloch, society’s definition of suffering could in fact invert, and some future Effective Altruism movement may very well work to maximize what we today would call suffering.

I believe I have a way out of this: I think consciousness and suffering(valence) are both ‘real’, and so a crisp definition of each exists, about which one can be correct or incorrect. My challenge to you is to find a way out of this ‘repugnant conclusion’ also. Or to disprove that I’ve found a way out of it, of course. :)

In short, I think we can be constructivists about qualia & suffering, or we can be very concerned about reducing suffering, but I question the extent to which we can do both at the same time while maintaining consistency.

Comment author: RomeoStevens 16 January 2017 09:09:59AM 1 point [-]

The frustrating inverse point makes me think this is a reflection of the asymmetric payoff structure in the AE.

Comment author: MikeJohnson 16 January 2017 07:54:32PM *  2 points [-]

Interesting- to attempt to re-state your notion: it's more important to avoid death than get an easy meal, so pain&aversion should come easier than pleasure.

I'd agree with this, but perhaps this is overdetermined in that both evolution and substrate lead us to "pleasure is centralized&highly contextual, pain is distributed&easily caused".

I.e., I would expect that given a set of conscious systems with randomized configurations, valence probably doesn't fall into a standard distribution. Rather, my expectation is that high-valence states will be outnumbered by low-valence states... and so, just like it's easier to destroy value than create it, it's easier to create negative valence than positive valence. Thus, positive valence requires centralized coordination (hedonic regions) and is easily disrupted by nociceptors (injections of entropy are unlikely to push the system toward positive states, since those are rare).

Comment author: Brian_Tomasik 16 January 2017 09:58:15AM 4 points [-]

Thanks for the summary! Lots of useful info here.

for every functional story about the role of valence, there exist counter-examples.

As a functionalist, I'm not at all troubled by these counter-examples. They merely show that the brain is very complicated, and they reinforce my view that crisp definitions of valence don't work. ;)

As an analogy, suppose you were trying to find the location of "pesticide regulation" in the United States. You might start with the EPA: "Pesticide regulation in the United States is primarily a responsibility of the Environmental Protection Agency." But you might notice that other federal agencies do work related to pesticides (e.g., the USDA). Moreover, some individual states have their own pesticide regulations. Plus, individual schools, golf courses, and homes decide if and how to apply pesticides; in this sense, they also "regulate" pesticide use. We might try to distinguish "legal regulation" from "individual choices" and note that the two can operate differently. We might question what counts as a pesticide. And so on. All this shows is that there's a lot of stuff going on that doesn't cleanly map onto simple constructs.

Actually, your later Barrett (2006) quote says the same thing: “the natural-kind view of emotion may be the result of an error of arbitrary aggregation. That is, our perceptual processes lead us to aggregate emotional processing into categories that do not necessarily reveal the causal structure of the emotional processing.” And you seemed to agree in your conclusion: "valence in the human brain is a complex phenomenon which defies simple description." I'm puzzled how this squares with your attempt to find a crisp definition for valence.

we don’t have a clue as to what properties are necessary or sufficient to make a given brain region a so-called “pleasure center” or “pain center”

Likewise, we can debate the necessary and sufficient properties that make something a "pesticide-regulation center".

by taking a microprocessor [...] and attempting to reverse-engineer it

Interesting. :) This is part of why I don't expect whole-brain emulation to come before de-novo AGI. Reverse-engineering of complex systems is often very difficult.

Comment author: MikeJohnson 16 January 2017 07:27:06PM *  3 points [-]

Hi Brian,

Thanks for the thoughts & kind words.

Nominally, this post is simply making the point that affective neuroscience doesn't have a good definition of valence nor suffering, and based on its current trajectory, isn't likely to produce one in the foreseeable future. It seems we both agree on that. :) However, you're quite correct that the subtext to this post is that I believe a crisp definition of valence is possible, and you're curious how I square this with the above description of the sad state of affective neuroscience.

Essentially, my model is that valence in the human brain is an incredibly complex phenomenon that defies simple description-- but valence itself is probably a simple property of conscious systems. This seems entirely consistent with the above facts (Section I of my paper), and also very plausible if consciousness is a physical phenomenon. Here are the next few paragraphs of my paper:

II. Clarifying the Problem of Valence

The above section noted that affective neuroscience knows a lot about valence, but its knowledge is very messy and disorganized. If valence is intrinsically a messy, fuzzy property of conscious states, perhaps this really is the best we can do here.

However, I don’t think we live in a universe where valence is a fuzzy, fragile, high-level construction. Instead, I think it’s a crisp thing we can quantify, and the patterns in it only look incredibly messy because we’re looking at it from the wrong level of abstraction.

Brains vs conscious systems:

There are fundamentally two kinds of knowledge about valence: things that are true specifically in brains like ours, and general principles common to all conscious entities. Almost all of what we know about pain and pleasure is of the first type-- essentially, affective neuroscience has been synonymous with making maps of the mammalian brain’s evolved, adaptive affective modules and contingent architectural quirks (“spandrels”).

This paper attempts to chart a viable course for this second type of research: it’s an attempt toward a general theory of valence, a.k.a. universal, substrate-independent principles that apply equally to and are precisely true in all conscious entities, be they humans, non-human animals, aliens, or conscious artificial intelligence (AI).

...

Anything piped through the complexity of the brain will look complex, regardless of how simple or complex it starts out as. Similarly, anything will look irreducibly complex if we're looking at it from the wrong level of abstraction. So just because affective neuroscience is confused about valence, doesn't mean that valence is somehow intrinsically confusing.

In this sense, I see valence research as no different than any other physical science: progress will be made by (1) controlling for the messy complexity added by studying valence in messy systems, and (2) finding levels of abstractions that "carve reality at the joints" better. (For instance, "emotions" are not natural kinds, as Barrett notes, but "valence" may be one.)

The real kicker here is whether there exists a cache of predictive knowledge about consciousness to be discovered (similar to how Faraday&Maxwell discovered a cache of predictive knowledge about electromagnetism) or whether consciousness is a linguistic confusion, to be explained away (similar to how elan vital was a linguistic confusion & improper reification).

Fundamental research about suffering looks very, very different depending on which of these is true. Principia Qualia lays out how it would look in the case of the former, and describes a research program that I expect to bear predictive fruit if we 'turn the crank' on it.

But there doesn't seem to be an equivalent document describing what suffering research is if we assume that consciousness should be thought of more as a linguistic confusion than a 'real' thing, and that suffering is a leaky reification. Explicitly describing what fundamental research about suffering looks like, and predicting what kinds of knowledge are & aren't possible, if we assume functionalism (or perhaps 'computational constructivism' fits your views?) seems like it could be a particularly worthwhile project for FRI.

p.s. Yes, I quite enjoyed that piece on attempting to reverse-engineer a 6502 microprocessor via standard neuroscientific methods. My favorite paper of 2016 actually!

10

A review of what affective neuroscience knows about suffering & valence. (TLDR: Affective Neuroscience is very confused about what suffering is.)

Part 1 A significant fraction of the EA movement is concerned with suffering, and all else being equal, think there should be less of it. I think this is an extraordinarily noble goal.  But what *is* suffering? There are roughly as many working definitions of suffering in the EA movement... Read More

View more: Next