Hi, all! The Machine Intelligence Research Institute (MIRI) is answering questions here tomorrow, October 12 at 10am PDT. You can post questions below in the interim.
MIRI is a Berkeley-based research nonprofit that does basic research on key technical questions related to smarter-than-human artificial intelligence systems. Our research is largely aimed at developing a deeper and more formal understanding of such systems and their safety requirements, so that the research community is better-positioned to design systems that can be aligned with our interests. See here for more background.
Through the end of October, we're running our 2016 fundraiser — our most ambitious funding drive to date. Part of the goal of this AMA is to address questions about our future plans and funding gap, but we're also hoping to get very general questions about AI risk, very specialized questions about our technical work, and everything in between. Some of the biggest news at MIRI since Nate's AMA here last year:
- We developed a new framework for thinking about deductively limited reasoning, logical induction.
- Half of our research team started work on a new machine learning research agenda, distinct from our agent foundations agenda.
- We received a review and a $500k grant from the Open Philanthropy Project.
Likely participants in the AMA include:
- Nate Soares, Executive Director and primary author of the AF research agenda
- Malo Bourgon, Chief Operating Officer
- Rob Bensinger, Research Communications Manager
- Jessica Taylor, Research Fellow and primary author of the ML research agenda
- Tsvi Benson-Tilsen, Research Associate
Nate, Jessica, and Tsvi are also three of the co-authors of the "Logical Induction" paper.
EDIT (10:04am PDT): We're here! Answers on the way!
EDIT (10:55pm PDT): Thanks for all the great questions! That's all for now, though we'll post a few more answers tomorrow to things we didn't get to. If you'd like to support our AI safety work, our fundraiser will be continuing through the end of October.
The ideal MIRI researcher is someone who’s able to think about thorny philosophical problems and break off parts of them to formalize mathematically. In the case of logical uncertainty, researchers started by thinking about the initially vague problem of reasoning well about uncertain mathematical statements, turned some of these thoughts into formal desiderata and algorithms (producing intermediate possibility and impossibility results), and eventually found a way to satisfy many of these desiderata at once. We’d like to do a lot more of this kind of work in the future.
Probably the main difference between MIRI research and typical AI research is that we focus on problems of the form “if we had capability X, how would we achieve outcome Y?” rather than “how can we build a practical system achieving outcome Y?”. We focus less on computational tractability and more on the philosophical question of how we would build a system to achieve Y in principle, given e.g. unlimited computing resources or access to extremely powerful machine learning systems. I don’t think we have much special knowledge that others don’t have (or vice versa), given that most relevant AI research is public; it’s more that we have a different research focus that will lead us to ask different questions. Of course, our different research focus is motivated by our philosophy about AI, and we have significant philosophical differences with most AI researchers (which isn’t actually saying much given how much philosophical diversity there is in the field of AI).
Work in the field of AI can inform us about what approaches are most promising (e.g., the theoretical questions in the “Alignment for Advanced Machine Learning Systems” agenda are of more interest if variants of deep learning are sufficient to achieve AGI), and can directly provide useful theoretical tools (e.g., in the field of statistical learning theory). Typically, we will want to get a high-level view of what the field is doing and otherwise focus mainly on the more theoretical work relevant to our research interests.
We definitely need some way of dealing with the fact that we don’t know which AI paradigm(s) will be the foundation of the first AGI systems. One strategy is to come up with abstractions that work across AI paradigms; we can ask the question “if we had access to extremely powerful reinforcement learning systems, how would we use them to safely achieve some concrete objective in the world?” without knowing how these reinforcement learning systems work internally. A second strategy is to prioritize work related to types of AI systems that seem more promising (deep learning seems more promising than symbolic GOFAI at the moment, for example). A third strategy is to do what people sometimes do when coming up with new AI paradigms: think about how good reasoning works, formalize some of these aspects, and design algorithms performing good reasoning according to these desiderata. In thinking about AI alignment, we apply all three of these strategies.