As far as I can tell, there seems to be a strong tendency for those who are worried about AI risk to also be longtermists.
However many elements of the philosophical case for longtermism are independent of contingent facts about what is going to happen with AI in the coming decades.
If we have good community epistemic health, we should expect there to be people who object to longtermism on grounds like:
- person-affecting views
- supporting a non-zero pure discount rate
but who still are just as worried about AI as those with P(doom) > 90%.
Indeed, the proportion of "doomers" with those philosophical objections to longtermism should be just as high as the rate of such philosophical objections among those typically considered neartermist.
I'm interested in answers either of the form:
- "hello, I'm both neartermist and have high P(doom) from AI risk..."; or
- "here's some relevant data, from, say, the EA survey, or whatever"
I think non-longtermists don't hold these premises; rather, they object to longtermism on tractability grounds.
My subjective impression was this actually applied to a reasonable number of people in the longtermist social cluster. I think Will's technical definition is just not that close a fit for the community.
I don't think we'll see this, largely because I expect having high AI x-risk estimates correlates with taking abstract arguments like longtermism seriously.
I'd also be pretty interested to get to know someone who thinks AI doom is inevitable and works to reduce suffering while we still have some power. I feel like some people who find AI alignment almost impossibly intractable should work on neartermist causes but I've never seen that happen.
I guess I am SORT OF one of those people you want to know.
I think AI non-allignment with humanity's interests is often very closely related to the non-allignment of capital investment and large corporate interests with humanity's interests.
That is because I think it is most likely that large capital investments will be creating the most powerful AGI in the future, and that investment is controlled by what the old Marxists call the class interest of international capital.
Thus, I think that it is inevitable that profit-maximizing AGI will have interests that diverge from humanity's interests to a greater and greater degree over time. This is similar to how the interests of the profit-maximizing wealthy humans diverge from the interests of the low-income global majority over time.
Thus the highly likely x-risk is that humans have little or no influence over society 100 to 1000 years from now. I think there is pretty close to zero chance that humans will become physically extinct (but this could be debated).
Thus I consider any neartermist work that strives to better align the interests of profit-maximizing wealthy humans with the interests of the current global low-income majority to be contributing strongly to the longtermist effort to align the interests of a future profit-maximizing AGI (which is likely to be the most powerful future AGI) with the interests of the majority of a future humanity.
Extrapolatinf wealth inequality trends, the vast majority of a future humanity is likely to be hundreds or thousands of times less wealthy than the most powerful AGIs.
This, my personal feeling is that neartermist "wealth inequality alignment" work is more likely to be effective than much longtermist work, because I think it is very likely that the vast majority of current longtermists have little or no verifiable knowledge about how to they can actually influence the nature or character of the future most powerful AGI's that are likely to dominate our future solar system. My personal "guess" is that it is those future AGIs that are likely to create the greatest risks for humanity.
Who these AGIs are or what they will be like, I don't think anyone really knows at the present time. This isn't because the problem is untractable in the classic sense, but it is because the chaotic system that will create the AGIs very likely has a very high internal dynamic instability that creates large sensitivity to initial conditions (which makes the result predictably unpredictable via the "butterfly effect").
I think that by working on neartermist problems, it is possible for us to change the societal conditions that are the initial conditions for the creation of future AGIs. I would argue that if we make our current human society such that humans are nicer to humans in the near term, then this will create a set of initial conditions where it is also more likely that future human-initiated AGI is nicer to humans in the future.
Thus, I think there is lots of neartermist work on wealth inequality that can potentially be very effective at preventing the future AGI x-risk of a completely marginalized humanity. I think that work is both more tractable, has far more certain results AND addresses some of our most probable longtermist x-risks.
Much of my past technical research was in the field of "Chaotic Dynamics" which I don't think has been applied to EA longtermist philosophy yet. My experience with the dynamics of complex systems makes me very skeptical about any forecasts of the particulars regarding the future of AGI and the agency that individuals are likely to have in creating that future.
That background is the technical context for my views.
Is that what you wanted to get to know?
I think a lot of people who are aware of AI risk for some time but nevertheless choose to work on some other causes, such as climate change, may implicitly hold this view.
Isn't the opposite end of the p(doom)–longtermism quadrant also relevant? E.g. my p(doom) is 2%, but I take the arguments for longtermism seriously and think that's high enough of a chance to justify working on the alignment problem.
interesting, I would instinctively still consider 2% to be a high p(doom) within the next 100 years. In the AI field, what is generally seen as a "high" or "low" p(doom)
It could be that both acceptance of longtermism and ability to forecast AI accurately are caused by some shared underlying factor, e.g. ability to reason and think systematically correctly (or incorrectly).
Or, put another way: in general, for any two questions where there is an objectively correct answer, giving correct answers should be pretty correlated, even if the questions themselves are completely unrelated. Forecasting definitely has an objectively correct answer; not sure about longtermism vs. neartermism, but I think it's plausible that it will one day look settled, or at least come down to mostly epistemic uncertainty.
So I don't see why views on these topics should be uncorrelated, unless you think philosophical questions about longtermism vs. neartermism are simple questions of opinion or values differences with no epistemic uncertainty left, and that peoples' answers to them are unlikely to change under reflection.