Andrew Critch's guest appearances

Andrew Critch

Future of Life Institute Podcast

Computing Up

AXRP - the AI X-risk Research Podcast

The Nonlinear Library

The Nonlinear Library: Alignment Forum Top Posts

The Nonlinear Library: EA Forum

LessWrong Curated Podcast

Sam Harris

Steven Pinker

Mike Johnson

Yuval Noah Harari

Fran Moore

Dario Amodei

Karen Levy

Robin Hanson

Stuart Russell

Sean Carroll

Stephen Batchelor

Glen Weyl

Melanie Mitchell

William MacAskill

Joshua Greene

OwlTail

Andrew Critch (🔗), a mathematician, AI researcher, organizer and activist (cofounder 🔗, researcher 🔗, cofounder 🔗),  joins Michael and Dave for a fast-moving fifty minutes about existential risks (and opportunities) of AI and other technologies, the limits of intelligence, and the importance of structure at all scales and having a good spirit.

Andrew Critch (<a href="https://acritch.com" rel="noopener nofollow" target="_blank">🔗</a>), a mathematician, AI researcher, organizer and activist (cofounder <a href="http://rationality.org/" rel="noopener nofollow" target="_blank">🔗</a>, researcher <a href="https://humancompatible.ai/" rel="noopener nofollow" target="_blank">🔗</a>, cofounder <a href="https://humancompatible.ai/" rel="noopener nofollow" target="_blank">🔗</a>),  joins Michael and Dave for a fast-moving fifty minutes about existential risks (and opportunities) of AI and other technologies, the limits of intelligence, and the importance of structure at all scales and having a good spirit.

Andrew Critch on AI - 56th Conversation

In this episode of the AI Alignment Podcast, Andrew Critch joins us to discuss a recent paper he co-authored with David Krueger titled AI Research Considerations for Human Existential Safety. We explore a wide range of issues, from how the mainstream computer science community views AI existential risk, to the need for more accurate terminology in the field of AI existential safety and the risks of what Andrew calls prepotent AI systems. Crucially, we also discuss what Andrew sees as being the most likely source of existential risk: the possibility of externalities from multiple AIs and AI stakeholders competing in a context where alignment and AI existential safety issues are not naturally covered by industry incentives. Topics discussed in this episode include:- The mainstream computer science view of AI existential risk- Distinguishing AI safety from AI existential safety - The need for more precise terminology in the field of AI existential safety and alignment- The concept of prepotent AI systems and the problem of delegation - Which alignment problems get solved by commercial incentives and which don’t- The threat of diffusion of responsibility on AI existential safety considerations not covered by commercial incentives- Prepotent AI risk types that lead to unsurvivability for humanityYou can find the page for this podcast here: https://futureoflife.org/2020/09/15/andrew-critch-on-ai-research-considerations-for-human-existential-safety/Timestamps: 0:00 Intro2:53 Why Andrew wrote ARCHES and what it’s about6:46 The perspective of the mainstream CS community on AI existential risk13:03 ARCHES in relation to AI existential risk literature16:05 The distinction between safety and existential safety 24:27 Existential risk is most likely to obtain through externalities 29:03 The relationship between existential safety and safety for current systems 33:17 Research areas that may not be solved by natural commercial incentives 51:40 What’s an AI system and an AI technology? 53:42 Prepotent AI 59:41 Misaligned prepotent AI technology 01:05:13 Human frailty 01:07:37 The importance of delegation 01:14:11 Single-single, single-multi, multi-single, and multi-multi 01:15:26 Control, instruction, and comprehension 01:20:40 The multiplicity thesis 01:22:16 Risk types from prepotent AI that lead to human unsurvivability 01:34:06 Flow-through effects 01:41:00 Multi-stakeholder objectives 01:49:08 Final words from AndrewThis podcast is possible because of the support of listeners like you. If you found this conversation to be meaningful or valuable, consider supporting it directly by donating at futureoflife.org/donate. Contributions like yours make these conversations possible.

In this episode of the AI Alignment Podcast, Andrew Critch joins us to discuss a recent paper he co-authored with David Krueger titled AI Research Considerations for Human Existential Safety. We explore a wide range of issues, from how the mainstream computer science community views AI existential risk, to the need for more accurate terminology in the field of AI existential safety and the risks of what Andrew calls prepotent AI systems. Crucially, we also discuss what Andrew sees as being the most likely source of existential risk: the possibility of externalities from multiple AIs and AI stakeholders competing in a context where alignment and AI existential safety issues are not naturally covered by industry incentives.  Topics discussed in this episode include: - The mainstream computer science view of AI existential risk - Distinguishing AI safety from AI existential safety  - The need for more precise terminology in the field of AI existential safety and alignment - The concept of prepotent AI systems and the problem of delegation  - Which alignment problems get solved by commercial incentives and which don’t - The threat of diffusion of responsibility on AI existential safety considerations not covered by commercial incentives - Prepotent AI risk types that lead to unsurvivability for humanity You can find the page for this podcast here: <a href="https://futureoflife.org/2020/09/15/andrew-critch-on-ai-research-considerations-for-human-existential-safety/" rel="nofollow" target="_blank">https://futureoflife.org/2020/09/15/andrew-critch-on-ai-research-considerations-for-human-existential-safety/</a> Timestamps:  0:00 Intro 2:53 Why Andrew wrote ARCHES and what it’s about 6:46 The perspective of the mainstream CS community on AI existential risk 13:03 ARCHES in relation to AI existential risk literature 16:05 The distinction between safety and existential safety  24:27 Existential risk is most likely to obtain through externalities  29:03 The relationship between existential safety and safety for current systems  33:17 Research areas that may not be solved by natural commercial incentives 51:40 What’s an AI system and an AI technology?  53:42 Prepotent AI  59:41 Misaligned prepotent AI technology  01:05:13 Human frailty  01:07:37 The importance of delegation  01:14:11 Single-single, single-multi, multi-single, and multi-multi  01:15:26 Control, instruction, and comprehension  01:20:40 The multiplicity thesis  01:22:16 Risk types from prepotent AI that lead to human unsurvivability  01:34:06 Flow-through effects  01:41:00 Multi-stakeholder objectives  01:49:08 Final words from Andrew This podcast is possible because of the support of listeners like you. If you found this conversation to be meaningful or valuable, consider supporting it directly by donating at <a href="http://futureoflife.org/donate" rel="nofollow" target="_blank">futureoflife.org/donate</a>. Contributions like yours make these conversations possible.

Andrew Critch on AI Research Considerations for Human Existential Safety

Anthony Aguirre

How well can we predict the future? In this podcast, Ariel speaks with Anthony Aguirre and Andrew Critch about the art of predicting the future, what constitutes a good prediction, and how we can better predict the advancement of artificial intelligence. They also touch on the difference between predicting a solar eclipse and predicting the weather, what it takes to make money on the stock market, and the bystander effect regarding existential risks.Visit metaculus.com to try your hand at the art of predicting.Anthony is a professor of physics at the University of California at Santa Cruz. He's one of the founders of the Future of Life Institute, of the Foundational Questions Institute, and most recently of Metaculus.com, which is an online effort to crowdsource predictions about the future of science and technology. Andrew is on a two-year leave of absence from MIRI to work with UC Berkeley's Center for Human Compatible AI. He cofounded the Center for Applied Rationality, and previously worked as an algorithmic stock trader at James Street Capital.

How well can we predict the future? In this podcast, Ariel speaks with Anthony Aguirre and Andrew Critch about the art of predicting the future, what constitutes a good prediction, and how we can better predict the advancement of artificial intelligence. They also touch on the difference between predicting a solar eclipse and predicting the weather, what it takes to make money on the stock market, and the bystander effect regarding existential risks. Visit <a href="http://metaculus.com" rel="nofollow" target="_blank">metaculus.com</a> to try your hand at the art of predicting. Anthony is a professor of physics at the University of California at Santa Cruz. He's one of the founders of the Future of Life Institute, of the Foundational Questions Institute, and most recently of <a href="http://Metaculus.com" rel="nofollow" target="_blank">Metaculus.com</a>, which is an online effort to crowdsource predictions about the future of science and technology. Andrew is on a two-year leave of absence from MIRI to work with UC Berkeley's Center for Human Compatible AI. He cofounded the Center for Applied Rationality, and previously worked as an algorithmic stock trader at James Street Capital.

The Art Of Predicting With Anthony Aguirre And Andrew Critch