OCB

Owen Cotton-Barratt

8360 karmaJoined Aug 2014

Sequences
2

On Wholesomeness
Everyday Longermism

Comments
748

Topic contributions
3

Yeah I'm arguing that with good reflective governance we should achieve a large fraction of what's accessible.

It's quite possible that that means "not quite all", e.g. maybe there are some trades so that we don't aestivate in this galaxy, but do in the rest of them; but on the aggregative view that's almost as good as aestivating everywhere.

I think in this context the natural way to interpret probabilities (or medians) of probabilities is as something like: "what would I think the probability was if I knew a lot more about the situation and got to think about it a lot more, but wasn't literally omniscient". Of course that isn't fully defined (since it's not specified exactly how much more they get to know), but I think it's approximately meaningful, and can capture something quite useful/important about credal resilience.

Relatedly, I think the journalist's presentation is misleading. I think it makes it sound like there's something like an objective 1-in-230 chance of catastrophe. I think a more accurate presentation would be "expert thinks there's a 1% chance we live in a world with a high risk". This changes the appropriate policy implications: if taken at face value they both suggest there's something important to do, but in the latter case the action is less like "work out what to do about simulation shutdown" and more like "start a serious research programme to work out how likely it is we're in the fraction of worlds that was offhand labelled as 1%".

(I also think that you may be right that the cosmologist didn't really believe their numbers. I think that issue alone would be enough reason to be suspicious of the journalist's approach. They might reasonably suspect that such off-the-cuff answers might not withstand scrutiny, so it seems bad to report them in a way that could sound like considered expert opinion.)

I think maybe yes? But I'm a bit worried that "won't react to them" is actually doing a lot of work.

We could chat about more a concrete example that you think fits this description, if you like.

Re:

I just notice that I have a strong intuition, backed up by something that seems to me like a plausible claim: given that myriad actors always contribute to any outcome, it is hard to imagine that there is one (or a very few) individual(s) that does all of the heavy lifting...

I want to note that this property isn't a consequence of a power-law distribution. (It's true of some power laws but not others, depending on the exponent.) I think you're right about this in most cases (though in some domains like theoretical physics I think it's more plausible that most of the heavy lifting gets done by a few people). 

But even if there aren't a small number of individuals doing all the heavy lifting, it can still be the case that some people are doing far more than others. For example think of income distribution: it definitely isn't the case that just a few people earn most of the money, but it definitely is the case that some people earn far more than others. If you were advising someone on how to make as much money as possible, you wouldn't tell them to chase after the possibility that they could be in the 0.0001%, but you would want them to have an awareness of the shape of the distribution, and some idea of how to find high-paying industries; and if you were advising a lot of people you'd probably want to talk about circumstances in which founding a company would make sense.

Yeah the tone makes sense for a personal blog (and in general the piece makes more sense for an audience who can mostly be expected to know Katja already).

I think it could have signalled more not-being-a-puff-piece by making the frame less centrally about Katja and more about the virtues you wanted to draw attention to. It's something like: those, rather than the person, are the proper object-of-consideration for these large internet audiences. Then you could also mention that the source of inspiration was the person.

Extremely relevant for my personal assessment!

For the social fabric stuff it seems more important whether it's legibly not a puff piece. Had I downvoted (and honestly I was closer to upvoting), the intended signal would have been something like "small ding for failing to adequately signal that it's not a puff piece" (such signalling is cheaper for things that actually aren't puff pieces, so asking for it is relatively cheap and does some work to maintain a boundary against actual puff pieces). It would have warranted a bigger ding if I'd thought it was a puff piece.

It's still possible I'm miscalibrated and this is transparently not a puff piece to ~everyone. (Although the voting pattern on my comment suggests that my feeling was not an outlier ... I guess this is unsurprising to me as one of the reasons I wrote the comment was seeing that your post had some downvotes and thinking it might be helpful to voice my guess about why.)

I think I was leaning into making my guess sound surprising there, and I had in mind something closer to 100 than 30; it might have been better to represent it as "about 100" or ">50" or something.

The fact that presidential terms are just 4 or 8 years does play into my thinking. For sure, they've typically done other meaningful stuff, but I don't think that typically has such a high impact ratio as their years as president. I generated my ratio by querying my brain for snap judgements about how big a deal it would seem to have [some numbers of presidents] [do a thing over their career] vs [some fraction of congress].

Anyway I could certainly be wrong here. I think it's possible I'm underestimating how big is the impact of having the mouthpiece of the presidency.

FWIW my guess is that if you compare (lifetime impact of president):(lifetime impact of average member of congress), the ratio would be <100 (but >30).

I'm definitely a little surprised to hear that you don't think that impact is power-law distributed at all, even ex post. I wonder if it's worth trying to get numerical about this, rather than talk qualitatively about "whether impact is power-law distributed". Because really it's the quantitative ratios that matter rather than the exact nature of the distribution out in the tails (e.g. I doubt the essential disagreement here is about whether it's a power law vs a lognormal).

If you restrict to people who are broadly trying to do good with their work (at least a little bit), I'd be interested if you would offer guesses about the ratios (ex post) in impact comparing someone at the 90th centile to e.g. someone at the 50th centile; someone at the 99th centile; someone at the 99.99th centile. (I think it's kind of hard to produce numbers for these things because of course there's massive amounts of uncertainty, but my guess is that these four points would be spread out by somewhere between 2 and 4 orders of magnitude.)

And how much spread do we need to get here in order to justify a lot of attention going into looking for tail-upsides? Of course the exact amount of effort that's appropriate will vary with what you think of these tails, but if you think that some of your options might be twice as good (in expectation) than others, that's already enough to justify a lot of attention trying to make sure you find the good ones.

Notes on why I tend to expect something like a power law:

  • Some of my reason is looking at (what I understand of) the historical distribution of impact. It's certainly a bit flatter than a naive analysis would suggest after accounting for a bunch of the credit-sharing issues, selection effects in what we hear about, etc.; but I still think it will go like something along these lines.
  • Some of my reason is looking at distributions for some related things (like job productivity for jobs of various levels of complexity).
  • Some of my reason is having looked into the generating mechanisms for power laws and thinking that this looks like the type of place that they come up. (But this isn't super informative about numerically how thick the tails should be.)

I mostly-disagree with this on pragmatic grounds. I agree that that's the right approach to take on the first point if/when you have full information about what's going on. But in practice you essentially never have proper information on what everyone else's counterfactuals would look like according to different actions you could take.

If everyone thinks in terms of something like "approximate shares of moral credit", then this can help in coordinating to avoid situations where a lot of people work on a project because it seems worth it on marginal impact, but it would have been better if they'd all done something different. Doing this properly might mean impact markets (where the "market" part works as a mechanism for distributing cognition, so that each market participant is responsible for thinking through their own alternative options, and feeding that information into the system via their willingness to do work for different amounts of pay), but I think that you can get some rough approximation to the benefits of impact markets without actual markets by having people do the things they would have done with markets -- and in this context, that means paying attention to the share of credit different parties would get.

Load more