Comment author: tjmather  (EA Profile) 21 January 2017 07:36:21PM 0 points [-]

Absolutely yes, Against Malaria Foundation is very good from a human rights point of view :)

Comment author: Brian_Tomasik 21 January 2017 02:36:02PM *  0 points [-]

Thanks for the reply. :)

People also have a right not to die, so perhaps one could claim that AMF is as good for human rights as family planning?

As far as future stability, it's plausible that family planning beats AMF, both because of resource shortages and because of the unwanted-children thing you mention. Of course, while future stability has many upsides, it also makes it more likely that (post-)humanity will spread suffering throughout the cosmos.

Comment author: Brian_Tomasik 21 January 2017 02:21:28PM *  0 points [-]

But following this analogy, I’m offering a potential equation for electricity, but you’re saying that electricity doesn’t have an equation

I haven't read your main article (sorry!), so I may not be able to engage deeply here. If we're trying to model brain functioning, then there's not really any disagreement about what success looks like. Different neuroscientists will use different methods, some more biological, some more algorithmic, and some more mathematical. Insofar as your work is a form of neuroscience, perhaps from a different paradigm, that's cool. But I think we disagree more fundamentally in some way.

if you argue that something is bad and we should work to reduce it, but also say there’s no correct definition for it and no wrong definition for it, what are you really saying?

My point is that your objection is not an obstacle to practical implementation of my program, given that, e.g., anti-pornography activism exists.

If you want a more precise specification, you could define suffering as "whatever Brian says is suffering". See "Brian utilitarianism".

we could claim that “protons oppress electrons” or “there’s injustice in fundamental physics” — but this is obviously nonsense

It's not nonsense. :) If I cared about justice as my fundamental goal, I would wonder how far to extend it to simpler cases. I discuss an example with scheduling algorithms here. (Search for "justice" in that interview.)

we can apply this linguistic construct of “suffering” to arbitrary contexts without it losing meaning

We do lose much of the meaning when applying that concept to fundamental physics. The question is whether there's enough of the concept left over that our moral sympathies are still (ever so slightly) engaged.

That definition doesn’t seem to leave much room for ethical behavior

In my interpretation, altruism is part of "psychological advantage", e.g., helping others because you want to and because it makes you feel better to do so.

I assume you don’t think it’s 100% arbitrary whether we say something is suffering or not

I do think it's 100% arbitrary, depending how you define "arbitrary". But of course I deeply want people to care about reducing suffering. There's no contradiction here.

in accordance with new developments in the foundational physics, but we’re unlikely to chuck quantum field theory in favor of some idiosyncratic theory of crystal chakras. If we discover the universe’s equation for valence, we’re unlikely to find our definition of suffering at the mercy of intellectual fads.

Quantum field theory is instrumentally useful for any superintelligent agent. Preventing negative valence is not. Even if the knowledge of what valence is remains, caring about it may disappear.

But I think that, unambiguously, cats being lit on fire is an objectively bad thing.

I don't know what "objectively bad" means.

slide into a highly Darwinian/Malthusian/Molochian context, then I fear that could be the end of value.

I'm glad we roughly agree on this factual prediction, even if we interpret "value" differently.

Comment author: MikeJohnson 20 January 2017 09:11:24PM *  1 point [-]

[constructivism implies that] society’s definition of suffering, and any institutions we build whose mission is to reduce suffering, will almost certainly be co-opted by future intellectual fashions

If by this you mean society's prevailing concepts and values, then yes. But everything is at the mercy of those. If reducing your precisely defined version of suffering falls out of fashion, it won't matter that it has a crisp definition. :)

Our definition of electricity may evolve over time, in accordance with new developments in the foundational physics, but we’re unlikely to chuck quantum field theory in favor of some idiosyncratic theory of crystal chakras. If we discover the universe’s equation for valence, we’re unlikely to find our definition of suffering at the mercy of intellectual fads.

And so, I fear that if we’re constructivists about suffering, then we should expect a very dark scenario: that society’s definition of suffering, and any institutions we build whose mission is to reduce suffering, will almost certainly be co-opted by future intellectual fashions. And, in fact, that given enough time and enough Moloch, society’s definition of suffering could in fact invert, and some future Effective Altruism movement may very well work to maximize what we today would call suffering.

Hm, that doesn't seem too likely to me (more likely is that society becomes indifferent to suffering), except if you mean that altruists might, e.g., try to maximize the amount of sentience that exists, which would as a byproduct entail creating tons of suffering (but that statement already describes many EAs right now).

I agree that this seems unlikely, but it seems like you grant that such a values—inversion is possible, and say that it wouldn’t be a bad thing, because there’s no fundamental moral truth (moral nihilism). But I think that, unambiguously, cats being lit on fire is an objectively bad thing. Even if time and Moloch happen to twist the definition of ‘suffering’ such that future utilitarian EAs want to tile the universe with burning cats, I completely reject that such an intellectual fashion could be right.

I think most people would strongly agree with this moral realist position, rather than the moral nihilist position- that this specific thing is actually and unambiguously is bad, and that any definition of suffering that wouldn’t say it’s bad is wrong.

I think your solution, even if true, doesn't necessarily help with goal drift / Moloch stuff because people still have to care about the kind of suffering you're talking about. It's similar to moral realism: even if you find the actual moral truth, you need to get people to care about it, and most people won't (especially not future beings subject to Darwinian pressures).

Yeah, I mostly agree with this— Andres covers some of this with this post. I feel great urgency to figure this out while we’re still in the non-malthusian time Robin Hanson calls the dreamtime. If we don’t figure out what has value and slide into a highly Darwinian/Malthusian/Molochian context, then I fear that could be the end of value.

Comment author: Robert_Wiblin 20 January 2017 08:54:42PM 0 points [-]

"WE WILL BE TRUSTED TO THE EXTENT WE RESPECT THE STANDARD SOCIETAL NOTIONS OF INTEGRITY AND TRUST"

I think there is a lot to this, but I feel it can be subsumed into Paul's rule of thumb:

  • You should follow a standard societal notion of what is decent behaviour (unless you say ahead of time that you won't in this case) if you want people to have always thought that you are the kind of person who does that.

Because following standard social rules that everyone assumes to exist is an important part of being able to coordinate with others without very high communication and agreement overheads, you want to at least meet that standard (including following some norms you might have reservations about). Of course this doesn't preclude you meeting a higher standard if having a reputation for going above and beyond would be useful to you (as Paul argues it often is for most of us).

Comment author: MikeJohnson 20 January 2017 08:44:18PM 1 point [-]

I was worried about the same in reverse. I didn't find your comments rude. :)

Good! I’ll charge forward then. :)

As long as we're both using the same equations of physics to describe the phenomenon, it seems that exactly how we define "electricity" may not matter too much. The most popular interpretation of quantum mechanics is "shut up and calculate".

That is my favorite QM interpretation! But following this analogy, I’m offering a potential equation for electricity, but you’re saying that electricity doesn’t have an equation because it’s not ‘real’, so it doesn’t seem like you will ever be in a position to calculate.

“Well, blegblarg doesn’t have a crisp definition, it’s more of a you-know-it-when-you-see-it thing where there’s no ‘correct’ definition of blegblarg and we can each use our own moral compass to determine if something is blegblarg, but there’s definitely a lot of it out there and it’s clearly bad so we should definitely work to reduce it!”

Replace "blegblarg" with "obscenity", and you have an argument that many people suffering from religious viruses would endorse.

But that doesn’t address the concern: if you argue that something is bad and we should work to reduce it, but also say there’s no correct definition for it and no wrong definition for it, what are you really saying? You note elsewhere that “We can interpret any piece of matter as being conscious if we want to,” and imply something similar about suffering. I would say that a definition that allows for literally anything is not a definition; an ethics that says something is bad, but notes that it’s impossible to ever tell whether any particular thing is bad, is not an ethics.

An example I like to use is "justice". It's clear to many people that injustice is bad, even though there's no crisp, physics-based definition of injustice.

This doesn’t seem to match how you use the term ‘suffering’ in practice. E.g., we could claim that “protons oppress electrons” or “there’s injustice in fundamental physics” — but this is obviously nonsense, and from a Wittgensteinian “language game” point of view, what's happening is that we’re using perfectly good words in contexts where they break down. But you do want to say that there could be suffering in fundamental physics, and potentially the far future. It looks like you want to have your cake and eat it too, and say that (1) “suffering” is a fuzzy linguistic construct, like “injustice” is, but also that (2) we can apply this linguistic construct of “suffering” to arbitrary contexts without it losing meaning. This seems deeply inconsistent.

But you’re clearly not a moral nihilist

I am. :) At least by this definition: "Moral nihilists consider morality to be constructed, a complex set of rules and recommendations that may give a psychological, social, or economical advantage to its adherents, but is otherwise without universal or even relative truth in any sense."

That definition doesn’t seem to leave much room for ethical behavior (or foundational research!), merely selfish action. This ties into my notion above, that you seem to have one set of stated positions (extreme skepticism & constructivism about qualia & suffering, moral nihilism for the purpose of ‘psychological, social, or economical advantage’), but show different revealed preferences (which seem more altruistic, and seem to assume something close to moral realism).

The challenge in this space of consciousness/valence/suffering research is to be skeptical-yet-generative: to spot and explain the flaws in existing theories, yet also to constantly search for and/or build new theories which have the potential to avoid these flaws.

You have many amazing posts doing the former (I particularly enjoyed this piece ) but you seem to have given up on the latter, and at least in these replies, seem comfortable with extreme constructivism and moral nihilism. However, you also seem to implicitly lean on valence realism to avoid biting the bullet on full-out moral nihilism & constructivism— your revealed preferences seem to be that you still want meaning, you want to say suffering is actually bad, I assume you don’t think it’s 100% arbitrary whether we say something is suffering or not. But these things are not open to a full-blown moral nihilist.

Anyway, perhaps you would have very different interpretations on these things. I would expect so. :) I'm probing your argument to see what you do think. But in general, I agree with the sentiments of Scott Aaronson:

Yes, it’s possible that things like the hard problem of consciousness, or the measurement problem in quantum mechanics, will never have a satisfactory resolution. But even if so, building a complicated verbal edifice whose sole purpose is to tell people not even to look for a solution, to be satisfied with two “non-overlapping magisteria” and a lack of any explanation for how to reconcile them, never struck me as a substantive contribution to knowledge. It wasn’t when Niels Bohr did it, and it’s not when someone today does it either.

I want a future where we can tell each other to “shut up and calculate”. You may not like my solution for grounding what valence is (though I’m assuming you haven’t read Principia Qualia yet), but I hope you don’t stop looking for a solution.

Comment author: Paul_Christiano 20 January 2017 06:58:42PM 0 points [-]

Ah, that makes a lot more sense, sorry for misinterpreting you. (I think Toby has a view closer to the one I was responding to, though I suspect I am also oversimplifying his view.)

I agree that there are important philosophical questions that bear on the goodness of building various kinds of (unaligned) AI, and I think that those questions do have impact on what we ought to do. The biggest prize is if it turns out that some kinds of unaligned AI are much better than others, which I think is plausible. I guess we probably have similar views on these issues, modulo me being more optimistic about the prospects for aligned AI.

I don't think that an understanding of qualia is an important input into this issue though.

For example, from a long-run ethical perspective, whether or not humans have qualia is not especially important, and what mostly matters is human preferences (since those are what shape the future). If you created a race of p-zombies that nevertheless shared our preferences about qualia, I think it would be fine. And "the character of human preferences" is a very different kind of object than qualia. These questions are related in various ways (e.g. our beliefs about qualia are related to our qualia and to philosophical arguments about consciousness), but after thinking about that a little bit I think it is unlikely that the interaction is very important.

To summarize, I do agree that there are time-sensitive ethical questions about the moral value of creating unaligned AI. This was item 1.2 in this list from 4 years ago. I could imagine concluding that the nature of qualia is an important input into this question, but don't currently believe that.

Comment author: David_Althaus 20 January 2017 05:53:16PM 0 points [-]

Otoh, a few decades later handwashing did become mainstream. So I'd think that correct and clearly useful models have a great advantage in becoming adopted eventually. Good strategy/movement building is more relevant for hastening the rate of adoption.

To take another example: Communism profited from extremely good strategy/movement building at the beginning (Engels being one of the first EtGlers ever). But it ultimately failed to become widely accepted because it brought about bad consequences. Admittedly, it's still pretty popular, probably because it appeals to human intuitions (such as anti-market bias, etc.)

Comment author: tjmather  (EA Profile) 20 January 2017 02:29:45PM *  1 point [-]

I agree that contraceptives could increase wild-animal suffering in the short run. The challenge I've run into is how to balance the increase in short term wild-animal suffering against the rights of people to plan their pregnancies, as well as considerations around farm animal suffering. I feel a lot of uncertainty around this, and not sure we can definitively answer that question without having a better understanding of how much insects and other wild animals suffer.

I think what tips the balance for me is that I have the intuition that preventing unwanted pregnancies may increase world stability in the long run, which could lead to better outcomes in the future, since we'll have the luxury to be able to tackle stuff like wild animal suffering.

There is some evidence from a study in Europe that suggests that unwanted children have greater proneness to social problems and criminal activity. Another much more speculative consideration is whether there could be future conflicts related to resources such as water tables and topsoil being depleted around the world, depending if technology to produce food continues to keep up with the increasing demand for food.

In summary, I feel uncertain if contraceptives are a net positive or negative from a utilitarian point of view, but I do feel from a human rights point of view, that every pregnancy should be wanted.

Comment author: tjmather  (EA Profile) 20 January 2017 02:11:57PM 0 points [-]

That is a good point about the need for studies to measure the long term impact. What do you think of United Poultry Concerns? Do you know of any people who have given up chicken for ethical reasons but still eat other meat?

Do you think that cultured meat research should focus on developing alternatives to chicken instead of beef?

Comment author: Kathy 20 January 2017 02:09:01PM *  0 points [-]

I agree that most people will not understand the most strange ideas until they understand the basic ideas. Ensuring they understand the foundation is a good practice.

I definitely agree that the instances of weirdness that are beneficial are only a tiny fraction of the weirdness that is present.

Regarding weirdness:

There are effective and ineffective ways to be weird.

There are several apparently contradictory guidelines in art: "use design principles", "break the conventions", and "make sure everything looks intentional".

The effective ways to be weird manage all three guidelines.

Examples: Picasso, Björk, Lady Gaga

One of the major and most observable differences between these three artists vs. many weird people is that the behavior of the artists can be interpreted as a communication about something specific, meaningful, and valuable. Art is a language. Everything strange we do speaks about us. If you haven't studied art, it might be rather hard to interpret the above three artists. The language of art is sometimes completely opaque to non-artists, and those who interpret art often find a variety of different meanings rather than a consistent one. (I guess that's one reason why they don't call it science.) Quick interpretations: In Picasso, I interpret an exploration of order and chaos. In Björk, I interpret an exploration of the strangeness of nature, the familiarity and necessity of nature, and the contradiction between the two. In Lady Gaga, I interpret an edgy exploration of identity.

These artists have the skill to say something of meaning as they follow principles and break conventions in a way that looks intentional. That is why art is a different experience from, say, looking at an odd-shaped mud splatter on the sidewalk, and why it can be a lot more special.

Ineffective weirdness is too similar to the odd-shaped mud splatter. There need to be signs of intentional communication. To interpret meaning, we need to see that combination of unbroken principles and broken conventions arranged in an intentional-looking pattern.

Comment author: Kathy 20 January 2017 02:03:57PM *  0 points [-]

Edit: I agree that there aren't a large number of people advocating for dishonesty. My concern is that if even a small number of EAs get enough attention for doing something dishonest, this could cause us all reputation problems. Since we could be "painted with the same brush" due to the common human bias called stereotyping bias, I think it's worthwhile to make sure it's easy to find information about how to do honest promotion, and why.

I updated my post to mention some specific examples of the problems I've been seeing. Thank you, David.

Comment author: Telofy  (EA Profile) 20 January 2017 11:14:58AM 0 points [-]

Agreed wrt. honesty. (I’m from Germany.)

That weirdness is costly, though, is something that I’ve often heard and adopted myself, e.g., by asking friends how I can dress less weird and things like that. There’s also the typical progression that I’ve only heard challenged last year that you should first talk with people about poverty alleviation, and only when they understand basics like cost-effectiveness, triage, expected value, impartiality, etc., you can gradually lower your guard and start mentioning other animals and AIs.

Maybe Kathy doesn’t even contradict that, since the instances of weirdness that are beneficial may be a tiny fraction of all the weirdnesses that surrounds us, and finding out which tiny fraction it is (as well as employing it) will require that we first dial back all weirdnesses except for one candidate weirdness. I should just read that book.

Comment author: Brian_Tomasik 20 January 2017 10:48:11AM *  0 points [-]

So I would say very strongly that we can’t both say that electricity is subjective and everyone can have their own arbitrary poetic definition of what it is and how it works, but also do interesting and useful things with it.

As long as we're both using the same equations of physics to describe the phenomenon, it seems that exactly how we define "electricity" may not matter too much. The most popular interpretation of quantum mechanics is "shut up and calculate".

As another analogy, "life" has a fuzzy, arbitrary boundary, but that doesn't prevent us from doing biology.

If someone makes a strong assertion that something is bad and that we should work to reduce its prevalence, then they’re also implying it’s real in a non-trivial sense; if something is not real, then it cannot be bad in an actionable sense.

An example I like to use is "justice". It's clear to many people that injustice is bad, even though there's no crisp, physics-based definition of injustice.

“Well, blegblarg doesn’t have a crisp definition, it’s more of a you-know-it-when-you-see-it thing where there’s no ‘correct’ definition of blegblarg and we can each use our own moral compass to determine if something is blegblarg, but there’s definitely a lot of it out there and it’s clearly bad so we should definitely work to reduce it!”

Replace "blegblarg" with "obscenity", and you have an argument that many people suffering from religious viruses would endorse.

But you’re clearly not a moral nihilist

I am. :) At least by this definition: "Moral nihilists consider morality to be constructed, a complex set of rules and recommendations that may give a psychological, social, or economical advantage to its adherents, but is otherwise without universal or even relative truth in any sense."

my apologies if anything I've said comes across as rude

I was worried about the same in reverse. I didn't find your comments rude. :)

Comment author: DavidNash 20 January 2017 10:24:04AM 6 points [-]

This may be a community based thing but I haven't seen anyone advocating for lying in the UK and haven't heard of it much online either apart from one persons experience in California.

I agree with all the examples you have and think everyone should learn more about honest persuasion, but I'm not sure the myths to be bust are with the EA community rather than some peoples perception of the community.

Comment author: TracyCornett 20 January 2017 07:42:49AM 0 points [-]

Further study is best option, you can also look online for suggestions.

Comment author: MikeJohnson 20 January 2017 03:01:53AM *  1 point [-]

Thanks for the thoughts! Here's my attempt at laying out a strong form of why I don't think constructivism as applied to ethics & suffering leads to productive areas:

Imagine someone arguing that electromagnetism was purely a matter of definitions- there’s no “correct” definition of electricity, so how one approaches the topic and which definition one uses is ultimately a subjective choice.

But now imagine they also want to build a transistor. Transistors are, in fact, possible, and so it turns out that there is a good definition of electricity, by way of quantum theory, and of course many bad ones that don’t ‘carve reality at the joints’.

So I would say very strongly that we can’t both say that electricity is subjective and everyone can have their own arbitrary poetic definition of what it is and how it works, but also do interesting and useful things with it.

Likewise, my claim is that we can be a subjectivist about qualia and about suffering and say that how we define them is rather arbitrary and ultimately subjective, or we can say that some qualia are better than others and we should work to promote more good qualia and less bad qualia. But I don’t think we can do both at the same time. If someone makes a strong assertion that something is bad and that we should work to reduce its prevalence, then they’re also implying it’s real in a non-trivial sense; if something is not real, then it cannot be bad in an actionable sense.

Imagine that tomorrow I write a strong denouncement of blegblarg on the EA forum. I state that blegblarg is a scourge upon the universe, and we should work to rid ourselves of it, and all right-thinking people should agree with me. People ask me, “Mike…. I thought your post was interesting, but…. what the heck is blegblarg??” - I respond that “Well, blegblarg doesn’t have a crisp definition, it’s more of a you-know-it-when-you-see-it thing where there’s no ‘correct’ definition of blegblarg and we can each use our own moral compass to determine if something is blegblarg, but there’s definitely a lot of it out there and it’s clearly bad so we should definitely work to reduce it!”

This story would have no happy ending. Blegblarg can’t be a good rallying cry, because I can’t explain what it is. I can’t say it’s good or bad in a specific actionable sense, for the same reason. One person’s blegblarg is another person’s blargbleg, you know? :)

I see a strict reading of the constructivist project as essentially claiming similar things about suffering, and ultimately leading to concluding that what is, and isn't, suffering, is fundamentally arbitrary-- i.e., it leads to post-modern moral nihilism. But you’re clearly not a moral nihilist, and FRI certainly doesn’t see itself as nihilist. In my admittedly biased view of the situation, I see you & FRI circling around moral realism without admitting it. :) Now, perhaps my flavor of moral realism isn’t to your liking- perhaps you might come to a completely different principled conclusion about what qualia & valence are. But I do hope you keep looking.

p.s. I tend to be very direct when speaking about these topics, and my apologies if anything I've said comes across as rude. I think we differ in an interesting way and there may be updates in this for both of us.

Comment author: Brian_Tomasik 19 January 2017 06:23:33PM *  1 point [-]

I'd suggest that if pleasure-ceptors are easy contextually habituated, they might not be pleasure-ceptors per se.

Not sure why that is unless you're just defining things that way, which is fine. :)

BTW, this page says

While large mechanosensory neurons such as type I/group Aß display adaptation, smaller type IV/group C nociceptive neurons do not. As a result, pain does not usually subside rapidly but persists for long periods of time; in contrast, one quickly stops receiving touch or sensory information if surroundings remain constant.


Arguably, we do see both of these things happen to some degree with regard to "pseudo-pleasure-ceptors" in the pelvis(?).

Yeah, as well as with various other addictions.

Comment author: Brian_Tomasik 19 January 2017 06:19:09PM 0 points [-]

we could use odd definitions of suffering to conclude that animals probably don’t feel pain, or that current chatbots can feel pain, or that the suffering which happens when a cis white man steps on a nail, is less than the suffering which happens when a bisexual black female steps on a nail, or vice-versa.

But doing so would amount to shifting the goalpost, which is a way of cheating at arguments whether there's a single definition of a word or not. :)

It's similar to arguments over abortion of very early embryos. One side calls a small clump of cells "a human life", and the other side doesn't. There's no correct answer; it just depends what you mean by that phrase. But the disagreement isn't rendered trivial by the lack of objectivity of a single definition.

will almost certainly be co-opted by future intellectual fashions

If by this you mean society's prevailing concepts and values, then yes. But everything is at the mercy of those. If reducing your precisely defined version of suffering falls out of fashion, it won't matter that it has a crisp definition. :)

some future Effective Altruism movement may very well work to maximize what we today would call suffering.

Hm, that doesn't seem too likely to me (more likely is that society becomes indifferent to suffering), except if you mean that altruists might, e.g., try to maximize the amount of sentience that exists, which would as a byproduct entail creating tons of suffering (but that statement already describes many EAs right now).

My challenge to you is to find a way out of this ‘repugnant conclusion’ also. Or to disprove that I’ve found a way out of it, of course. :)

I think your solution, even if true, doesn't necessarily help with goal drift / Moloch stuff because people still have to care about the kind of suffering you're talking about. It's similar to moral realism: even if you find the actual moral truth, you need to get people to care about it, and most people won't (especially not future beings subject to Darwinian pressures).

Comment author: MikeJohnson 19 January 2017 05:24:28PM 1 point [-]

Hmm-- I'd suggest that if pleasure-ceptors are easy contextually habituated, they might not be pleasure-ceptors per se.

(Pleasure is easily habituated; pain is not. This is unfortunate but seems adaptive, at least in the AE...)

My intuition is that if an organism did have dedicated pleasure-ceptors, it would probably immediately become its biggest failure-point (internal dynamics breaking down) and attack surface (target for others to exploit in order to manipulate behavior, which wouldn't trigger fight/flight like most manipulations do).

Arguably, we do see both of these things happen to some degree with regard to "pseudo-pleasure-ceptors" in the pelvis(?).

View more: Next