Biting bullets (accepting a moral argument that's intuitively unpleasant because one feels logically compelled to) is central to many people's experiences in EA. In fact, bullet-biting often resembles a contest to demonstrate one's commitment to EA. I have even heard someone brag of having “teeth of steel for biting bullets”.
However, the compulsion to bite bullets is at odds with moral anti-realism, another popular belief among EAs. If there is no objective morality, then there is no reason to prefer a principled, fundamentalist morality to an ad hoc, intuitionist morality. In fact, for the following reasons, it would be expected that bullet-biting does not accord with most people's experiences of morality.
Is morality law-like?
The paper “Is Life Law-Like?” criticizes the quest of biologists to derive laws for their observations, pointing out that evolution leads to ad hoc solutions and that biological processes occur probabilistically. So is morality closer to physics or biology? An empirical approach to morality would view it as stemming from evolutionary psychology, social structures, and historical serendipity. Psychology, sociology, and history are some of the only fields even less law-like than biology. Attempts to derive laws for morality are largely confined to recent Western history.
Scott Alexander argues for “high-energy ethics”, the idea that only through extreme thought experiments and edge cases can we discern the true nature of morality. The allusion to physics is no accident: it is the only discipline that thrives in such extremes. Applying such an approach to psychology would be absurd. For example, some psychologists believe that tall men are viewed as more attractive. One psychologist tries to disprove this by engineering a nine-foot-tall man, who is not viewed as more attractive. Of course, she didn't actually refute the hypothesis, only demonstrated that observations in the social sciences are context-dependent.
Optimizing for multiple values
A popular bullet to bite is an argument of the form “X is theft/rape/murder”, where X is an act that is widely believed to be morally acceptable but that has superficial similarity to a serious crime. This has been called the worst argument in the world. The reason that it's naive and often rejected is that serious crimes are typically viewed as immoral for multiple reasons. Murder is immoral because, among other reasons, it causes fear and suffering through the act itself, the victim had a desire to continue living, the victim would have had future positive experiences, and the death brings grief to family and friends. X typically has one or several of these characteristics but not all, so it is commonly judged to be not as bad as murder.
Morality becomes even more complex when it involves competing values. There is no inconsistency in believing that airports should X-ray luggage to reduce security risks and simultaneously believing that widespread surveillance of citizens is unjustified. One can value both security and privacy and believe that in some cases one outweighs the other. This point is often lost in bullet-biting morality, which views “inconsistency” as a product of hypocrisy and cowardice.
Optimizing for a single value when one has multiple values will almost always lead to the sacrifice of some values. Paying workers by the hour leads to slow work; paying by the task leads to shoddy work. Similarly, optimizing for naive definitions of utility will lead to paperclipping. For example, if we believe in one definition of utility, we may end up with a universe tiled with thermostats. (Each thermostat is “happy” because it is programmed to “want” the temperature of the cosmic background radiation.)
Conclusion
Moral anti-realism doesn't like neat conclusions: though there's no reason to favor biting bullets, there's no reason to disfavor it either. However, I have two pragmatic observations. First, the pressure to bite bullets (and the implication of irrationality if one does not) can be be off-putting to some EAs. Second, it may be easier to maintain commitments to beliefs if they are sincere, rather than conclusions that you grudgingly accept because you see no other choice.
"You generally can't determine any facts about morality by studying its psychology, genealogy and history in society, as those refer to how people act and moral philosophy refers to how they ought to act."
Moral anti-realists think that questions about how people ought to act are fundamentally confused. For an anti-realist, the only legitimate questions about morality are empirical. What do societies believe about morality? Why do we believe these things (from a social and evolutionary perspective)? We can't derive normative truth from these questions, but they can still be useful.
"An anti-realist can still have reasons to affirm a consistent view of morality"
Consistent is not the same as principled. Of course I believe in internal consistency. But principled morality is no more rational than unprincipled morality.
"I'm not aware of this being common. The LessWrong link doesn't seem to be relevant to legitimate moral philosophy. Can you give some examples?"
Some EAs argue that killing animals for meat is the moral equivalent of murder. There are other examples outside EA: abortion is murder, taxation is theft. Ask tumblr what currently counts as rape... Just because some of these views aren't taken seriously by moral philosophers doesn't mean they aren't influential and shouldn't be engaged with.
"You can add more and more values to patch the holes and build a really complicated multivariate utility function which might end up producing normal outputs, but at this point I would question why you're optimizing at all, when it looks like what you really want to do is use an intuitionist approach."
Correct, I don't think utility function approaches are any better than avoiding utility functions. However, people have many moral values, and under normal circumstances these may approximate utility functions.
"Yes, although most people, moral realists included, would affirm a fundamental difference between phenomenal consciousness and movements of simple systems."
Consequentialism would require building a definition of consciousness into the utility function. Many definitions of consciousness, such as "complexity" or "integration", would fall apart in extreme cases.
Anti-realists deny that there is such thing as true moral claims, but they don't think morality is fundamentally confused. There have been many anti-realist philosophers who have proposed some form of ethics: R.M. Hare, J.L. Mackie, the existentialists, etc.
... (read more)