DavidMoss comments on How scale is often misused as a metric and how to fix it - Effective Altruism Forum

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (15)

You are viewing a single comment's thread. Show more comments above.

Comment author: Maxdalton 30 January 2018 12:14:47PM 5 points [-]

[Writing personally] This post seems to argue (at least implicitly) that scale is bad if it is the only metric that is used to assess a cause area or intervention. I think this is clearly correct.

But I don't think anyone has used scale as the only metric: 80,000 Hours very explicitly modify it with other factors (neglectedness, solvability), to account for things like the "bottlenecking" you discuss.

There's a separate thing you could mean, which is "Scale, neglectedness, solvability is only one model for prioritisation. It's useful to have multiple different models for prioritising. One alternate model is to assess what the biggest bottleneck is for solving a problem." (Note that this does not really support the claim that scale is misused: it's just that other lenses are also useful.)

I respect the inclination to use multiple models, and I think that thinking in terms of bottlenecks is useful for e.g. organizational prioritization. I think it's harder to apply to cause prioritization because we face so many problems and potential solutions that it's hard to see what the bottlenecks are. It may be useful for prioritizing how to use resources to pursue an intervention, which seems to be how you are mostly using it in this case.

Overall, I worry that your title doesn't really reflect what you show in the text.

Comment author: DavidMoss  (EA Profile) 30 January 2018 06:33:33PM 3 points [-]

I didn't read the post as meaning either "scale is bad if it is the only metric that is used" or "Scale, neglectedness, solvability is only one model for prioritisation. It's useful to have multiple different models...."

When looking at scale in a scale, neglectedness, tractability, framework, it's true that the other factors can offset the influence of scale. e.g. if something is large in scale but intractable, the intractability counts against the cause being considered and at least somewhat offsets the consideration that the cause is large in scale. But this doesn't touch on the point this post makes, which is that looking at scale itself as a consideration, the 'total scale' may be of little or no relevance to the evaluation of the cause, and rather 'scale' is only of value up to a given bottleneck and of no value beyond that. I almost never see people talking of scale in this way in the context of a scale, neglectedness, tractability, framework: dividing up the total scale into tractable bits, less tractable bits and totally intractable bits. Rather, I more typically see people assigning some points for scale, evaluating tractability independently and assigning some points for that and evaluating neglectedness independently and assigning some points for that.

Comment author: Joey 04 February 2018 09:08:38PM 2 points [-]

Thanks, David. Your interpretation is indeed what I was trying to get across.

Comment author: Arepo 30 January 2018 08:27:33PM 1 point [-]

I read this the same way as Max. The issue of cost to solve (eg) all cases of malaria is really tractability, not scale. Scale is how many people would be helped (and to what degree) by doing so. Divide the latter by the former, and you have a sensible-looking cost-benefit analysis, (that is sensitive to the 'size and intensity of the problem', ie the former).

I do think there are scale-related issues with drawing lines between 'problems', though - if a marginal contribution to malaria nets now achieves twice as much good as the same marginal contribution would in 5 years, are combatting malaria now and combatting malaria in five years 'different problems', or do you just try to average out the cost-benefit ratio between somewhat arbitrary points (eg now and when the last case of malaria is prevented/cured). But I also think the models Max and Owen have written about on the CEA blog do a decent job of dealing with this kind of question.

Comment author: Halstead 02 February 2018 06:20:34PM 0 points [-]

Your argument does not suggest that there is a problem with the commonly used conception of scale, but rather with how it is combined with tractability and neglectedness. Thus, it does not support the claims made in the main piece.

Comment author: DavidMoss  (EA Profile) 03 February 2018 12:43:07AM 1 point [-]

I disagree on both counts. I think my comment is recapitulating the core claims of the main piece (and am pretty confident the author would agree).

In my comment I mention the total S/T/N framework only because MaxDalton suggested that when properly viewed within that framework, the concerns with 'scale' Joey raised, don't apply. I argued that that Joey's concerns apply even if you are applying the full S/T/N framework, but I don't think they apply only if you are applying the full framework.

Comment author: Halstead 03 February 2018 12:07:08PM 0 points [-]

OK, but then the issue is problem individuation, not the conception of scale used.