6

Status Regulation and Anxious Underconfidence

Previous: Against Modest Epistemology   I’ve now given my critique of modesty as a set of explicit doctrines. I’ve tried to give the background theory, which I believe is nothing more than conventional cynical economics, that explains why so many aspects of the world are not optimized to the limits... Read More
5

Against Modest Epistemology

Previous: Blind Empiricism   Modest epistemology doesn’t need to reflect a skepticism about causal models as such. It can manifest instead as a wariness about putting weight down on  one’s own causal models, as opposed to others'. In 1976, Robert Aumann demonstrated that two ideal Bayesian reasoners with the same priors... Read More
6

Blind Empiricism

Previous: Living in an Inadequate World   The thesis that needs to be contrasted with modesty is not the assertion that everyone can beat their civilization all the time. It’s not that we should be the sort of person who sees the world as mad and pursues the strategy of... Read More
5

Living in an Inadequate World

Previous: Moloch's Toolbox ( pt. 1 , pt. 2 )   Be warned: Trying to put together a background model like the one I sketched in the previous chapter is a pretty perilous undertaking, especially if you don’t have a professional economist checking your work at every stage. Suppose I... Read More
9

Moloch's Toolbox (2/2)

Previous: Moloch's Toolbox (1/2)   vii. Sticky traditions in belief-dependent Nash equilibria without common knowledge Cecie:  I could talk next about a tax system that makes it cheaper for corporations to pay for care instead of patients, and how that sets up a host of “decisionmaker is not the beneficiary”... Read More
12

Moloch's Toolbox (1/2)

Previous: An Equilibrium of No Free Energy   There’s a toolbox of reusable concepts for analyzing systems I would call “inadequate”—the causes of civilizational failure, some of which correspond to local opportunities to do better yourself. I shall, somewhat arbitrarily, sort these concepts into three larger categories:  Decisionmakers who are... Read More
18

An Equilibrium of No Free Energy

Previous :   Inadequacy and Modesty   I am now going to introduce some concepts that lack established names in the economics literature—though I don’t believe that any of the basic ideas are new to economics. First, I want to distinguish between the standard economic concept of efficiency (as in... Read More
24

Inadequacy and Modesty

I'm posting my short new book here for discussion: Inadequate Equilibria: Where and How Civilizations Get Stuck . First chapter below, with the rest to follow over the coming days.   This is a book about two incompatible views on the age-old question: “When should I think that I may... Read More
Comment author: EliezerYudkowsky 28 September 2016 10:10:28PM *  16 points [-]

The idea of running an event in particular seems misguided. Conventions come after conversations. Real progress toward understanding, or conveying understanding, does not happen through speakers going On Stage at big events. If speakers On Stage ever say anything sensible, it's because an edifice of knowledge was built in the background out of people having real, engaged, and constructive arguments with each other, in private where constructive conversations can actually happen, and the speaker On Stage is quoting from that edifice.

(This is also true of journal publications about anything strategic-ish - most journal publications about AI alignment come from the void and are shouting into the void, neither aware of past work nor feeling obliged to engage with any criticism. Lesser (or greater) versions of this phenomenon occur in many fields; part of where the great replication crisis comes from is that people can go on citing refuted studies and nothing embarrassing happens to them, because god forbid there be a real comments section or an email reply that goes out to the whole mailing list.)

If there's something to be gained from having national-security higher-ups understanding the AGI alignment strategic landscape, or from having alignment people understand the national security landscape, then put Nate Soares in a room with somebody in national security who has a computer science background, and let them have a real conversation. Until that real progress has already been made in in-person conversations happening in the background where people are actually trying to say sensible things and justify their reasoning to one another, having a Big Event with people On Stage is just a giant opportunity for a bunch of people new to the problem to spout out whatever errors they thought up in the first five seconds of thinking, neither aware of past work nor expecting to engage with detailed criticism, words coming from the void and falling into the void. This seems net counterproductive.

Comment author: Dale 03 December 2014 12:34:31AM 4 points [-]

Interesting history!

However, I think you are being unfair to MIRI. Eliezer was using the term as as far back as 2007, four years before you mention it first being used in Oxford. So it wasn't originated in Oxford. And given that many CEA members have read LessWrong, including Toby Ord, it's seems a stretch to even say it was independently re-invented.

Comment author: EliezerYudkowsky 31 May 2015 05:02:02AM *  10 points [-]

There's only so many things you can call it, and accidental namespace collisions / phrase reinventions aren't surprising. I was surprised when I looked back myself and noticed the phrase was there, so it would be more surprising if Toby Ord remembered than if he didn't. I'm proud to have used the term "effective altruist" once in 2007, but to say that this means I coined the term, especially when it was re-output by the more careful process described above, might be giving me too much credit - but it's still nice to have this not-quite-coincidental mention be remembered, so thank you for that!

View more: Next