Thanks for writing this! I had one thought regarding how relevant saying no to some of the technologies you listed is to AGI. 

In the case of nuclear weapons programs, the use of fossil fuels, CFCs, and GMOs, we actively used these technologies before we said no (FFs and GMOs we still use despite 'no', and nuclear weapons we have and could use at a moments notice). With AGI, once we start using them it might be too late. Geo-engineering experiments is the most applicable out of these, as we actually did say no before any (much?) testing was undertaken.

Very fair response, thanks!

I strongly agree that current LLM's don't seem to pose a risk of a global catastrophe, but I'm worried about what might happen when LLM's are combined with things like digital virtual assistants who have outputs other than generating text. Even if it can only make bookings, send emails, etc., I feel like things could get concerning very fast.

Is there an argument for having AI fail spectacularly in a small way which raises enough global concern to slow progress/increase safety work? I'm envisioning something like a LLM virtual assistant which leads to a lot of lost productivity and some security breaches but nothing too catastrophic, which makes people take AI safety seriously, slowing progress on more advanced AI, perhaps.

A complete spitball.

This is cool! I came across EA in early 2015, and I've sometimes been curious about what happened in the years before then. Books like The Most Good You Can Do sometimes incidentally give anecdotes, but I haven't seen a complete picture in one public place. Not to toot our own horn too much, but I wonder if there will one day be a documentary about the movement itself.

Thanks for the great question. I'd like to see more attempts to get legislation passed to lock in small victories. The Sioux Falls slaughterhouse ban almost passing gives me optimism for this. Although it seemed to be more for NIMBY reasons than for animal rights reasons, in some ways that doesn't matter. 

I'm also interested in efforts to maintain the lower levels of speciesism we see in children into their adult lives, and to understand what exactly drives that so we can incorporate it into outreach attempts targeted at adults. Our recent interview with Matti Wilks touches on this a little if you're interested.

Thank you for the feedback! I just wanted to let you know that while I haven't had time to write a proper response, I've read your feedback and will try to take it on board in my future work.

People more involved with X-risk modelling (and better at math) than I could better say whether this is better than existing tools for X-risk modelling, but I like it! I hadn't heard of the absorbing state terminology, that was interesting. When reading that, my mind goes to option value, or lack thereof, but that might not be a perfect analogy.

Regarding x-risks requiring a memory component, can you design Markov chains to have the memory incorporated?

Some possible cases where memory might be useful (without thinking about it too much) might be:

• How well past social justice movements went may have implications for the success of future movements that relate to X-risk
• The way in which problems end may have implications for future problems

Maybe this information can just be captured without memory anyway? 

Thanks for sharing, I'm looking forward to this! I'm particularly excited about the sections on measuring suffering and artificial suffering.

Thanks for sharing! I love seeing concrete roadmaps/plans for things like this, and think we should do it more.