Raymond Arnold's guest appearances

Raymond Arnold

Life After God's tracks

Bi Any Means Podcast

The Nonlinear Library

The Nonlinear Library: Alignment Forum Top Posts

Anthony Pinn

James Croft

Alix Jules

Mandisa Thomas

Juhem Navarro-Rivera

Cass Midgley

Conor Robinson

Pat Green

Galen Broaddus

Aaron Rabinowitz

Chris Johnson

Blake Richards

Nick Bostrom

Robin Hanson

Stuart Russell

OwlTail

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: What's Up With Confusingly Pervasive Consequentialism?, published by Raymond Arnold on January 20, 2022 on The AI Alignment Forum.Fictionalized/Paraphrased version of a real dialog between me and John Wentworth.Fictionalized Me: So, in the Eliezer/Richard dialogs, Eliezer is trying to get across this idea that consequentialism deeply permeates optimization, and this is important, and that's one reason why Alignment is Hard. But something about it is confusing and slippery, and he keeps trying to explain it and it keeps not-quite-landing.I think I get it, but I'm not sure I could explain it. Or, I'm not sure who to explain it to. I don't think I could tell who was making a mistake, where "consequentialism is secretly everywhere" is a useful concept for realizing-the-mistake.Fictionalized John: [stares at me]Me: Okay, I guess I'm probably supposed to try and explain this and see what happens.Me: Okay, so the part that's confusing here is that this is supposed to be something that Eliezer thinks thoughtful, attentive people like Richard (and Paul?) aren't getting, despite them having read lots of relevant material and paying attention and being generally on board with "alignment is hard."...so, what is a sort of mistake I could imagine a smart, thoughtful person who read the sequences making here?My Eliezer-model imagines someone building what they think is an aligned ML system. They've trained it carefully to do things they reflectively approve of, they've put a lot of work into making it interpretable and honest. This Smart Thoughtful Researcher has read the sequences and believes that alignment is hard and whatnot. Nonetheless, they'll have failed to really grok this "consequentialism-is-more-pervasive-and-important-than-you-think" concept. And this will cause doom when they try to scale up their project to accomplish something actually hard.I... guess what I think Eliezer thinks is that Thoughful Researcher isn't respecting inner optimizers enough. They'll have built their system to be carefully aligned, but to do anything hard, it'll end up generating inner-optimizers that aren't aligned, and the inner-optimizers will kill everyone.John: Nod. But not quite. I think you're still missing something.You're familiar with the arguments of convergent instrumental goals?Me: i.e. most agents will end up wanting power/resources/self-preservation/etc?John: Yeah.But not only is "wanting power and self preservation" convergently instrumental. Consequentialism is convergently instrumental. Consequentialism is a (relatively) simple, effective process for accomplishing goals, so things that efficiently optimize for goals tend to approximate it.Now, say there's something hard you want to do, like build a moon base, or cure cancer or whatever. If there were a list of all the possible plans that cure cancer, ranked by "likely to work", most of the plans that might work route through "consequentalism", and "acquire resources."Not only that, most of the plans route through "acquire resources in a way that is unfriendly to human values." Because in the space of all possible plans, while consequentialism doesn't take that many bits to specify, human values are highly complex and take a lot of bits to specify.Notice that I just said "in the space of all possible plans, here are the most common plans." I didn't say anything about agents choosing plans or acting in the world. Just listing the plans. And this is important because the hard part lives in the choosing of the plans.Now, say you build an oracle AI. You've done all the things to try and make it interpretable and honest and such. If you ask it for a plan to cure cancer, what happens?Me: I guess it gives you a plan, and... the plan probably routes through consequentialist agents acquiring power in an unfriendly wa...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: What's Up With Confusingly Pervasive Consequentialism?, published by Raymond Arnold on January 20, 2022 on The AI Alignment Forum. Fictionalized/Paraphrased version of a real dialog between me and John Wentworth. Fictionalized Me: So, in the Eliezer/Richard dialogs, Eliezer is trying to get across this idea that consequentialism deeply permeates optimization, and this is important, and that's one reason why Alignment is Hard. But something about it is confusing and slippery, and he keeps trying to explain it and it keeps not-quite-landing. I think I get it, but I'm not sure I could explain it. Or, I'm not sure who to explain it to. I don't think I could tell who was making a mistake, where "consequentialism is secretly everywhere" is a useful concept for realizing-the-mistake. Fictionalized John: [stares at me] Me: Okay, I guess I'm probably supposed to try and explain this and see what happens. Me: Okay, so the part that's confusing here is that this is supposed to be something that Eliezer thinks thoughtful, attentive people like Richard (and Paul?) aren't getting, despite them having read lots of relevant material and paying attention and being generally on board with "alignment is hard." ...so, what is a sort of mistake I could imagine a smart, thoughtful person who read the sequences making here? My Eliezer-model imagines someone building what they think is an aligned ML system. They've trained it carefully to do things they reflectively approve of, they've put a lot of work into making it interpretable and honest. This Smart Thoughtful Researcher has read the sequences and believes that alignment is hard and whatnot. Nonetheless, they'll have failed to really grok this "consequentialism-is-more-pervasive-and-important-than-you-think" concept. And this will cause doom when they try to scale up their project to accomplish something actually hard. I... guess what I think Eliezer thinks is that Thoughful Researcher isn't respecting inner optimizers enough. They'll have built their system to be carefully aligned, but to do anything hard, it'll end up generating inner-optimizers that aren't aligned, and the inner-optimizers will kill everyone. John: Nod. But not quite. I think you're still missing something. You're familiar with the arguments of convergent instrumental goals? Me: i.e. most agents will end up wanting power/resources/self-preservation/etc? John: Yeah. But not only is "wanting power and self preservation" convergently instrumental. Consequentialism is convergently instrumental. Consequentialism is a (relatively) simple, effective process for accomplishing goals, so things that efficiently optimize for goals tend to approximate it. Now, say there's something hard you want to do, like build a moon base, or cure cancer or whatever. If there were a list of all the possible plans that cure cancer, ranked by "likely to work", most of the plans that might work route through "consequentalism", and "acquire resources." Not only that, most of the plans route through "acquire resources in a way that is unfriendly to human values." Because in the space of all possible plans, while consequentialism doesn't take that many bits to specify, human values are highly complex and take a lot of bits to specify. Notice that I just said "in the space of all possible plans, here are the most common plans." I didn't say anything about agents choosing plans or acting in the world. Just listing the plans. And this is important because the hard part lives in the choosing of the plans. Now, say you build an oracle AI. You've done all the things to try and make it interpretable and honest and such. If you ask it for a plan to cure cancer, what happens? Me: I guess it gives you a plan, and... the plan probably routes through consequentialist agents acquiring power in an unfriendly wa...

AF - What's Up With Confusingly Pervasive Consequentialism? by Raymond Arnold

My guest for today musician Raymond Arnold. He is the creator of a secular solstice celebration that’s held every year around the time of the winter solstice worldwide. According to the official Secular Solstice website, “This holiday was created for people with a worldview rooted in both science, progress and compassion. Who want to make the world a better place, and who understand that sometimes this means learning new things that challenge your worldview. The Secular Solstice is a time when we tell inspiring stories that remind us we’re not alone. That challenge us to work to make a better future. But, crucially, those stories have a firm grounding in our latest, best understanding of the world.” So today we’re going to talk about how this celebration began, what a typical service looks like, and how people can participate.Links:•SecularSolstice.com: http://secularsolstice.com/ •Brighter Than Today soundtrack: https://humanistculture.bandcamp.com/album/brighter-than-today-a-secular-solstice •Anthony Magnabosco: https://www.youtube.com/user/magnabosco210 •ReasonCon3: http://reasonnc.com/ •The Biskeptical Podcast: http://www.spreaker.com/show/the-biskeptical-podcast•Dream Youth: http://dreamyouth.bandcamp.com •Asher Silberman: http://www.ashersilberman.com/ •My Twitter: http://twitter.com/tmamone •Bi Any Means on Twitter: http://twitter.com/bianymeanspod •Bi Any Means on Facebook: http://www.facebook.com/bianymeanspodcast •The Bi Any Means/Biskeptical Podcast Fans Facebook group: https://www.facebook.com/groups/179617892470906/ •Email: bianymeanspodcast@gmail.com •Listener line: 410-690-3558   •My Patreon page: http://www.patreon.com/tmamone •Bi Any Means Blog: http://freethoughtblogs.com/bianymeans/

My guest for today musician Raymond Arnold. He is the creator of a secular solstice celebration that’s held every year around the time of the winter solstice worldwide. According to the official Secular Solstice website, “This holiday was created for people with a worldview rooted in both science, progress and compassion. Who want to make the world a better place, and who understand that sometimes this means learning new things that challenge your worldview. The Secular Solstice is a time when we tell inspiring stories that remind us we’re not alone. That challenge us to work to make a better future. But, crucially, those stories have a firm grounding in our latest, best understanding of the world.” So today we’re going to talk about how this celebration began, what a typical service looks like, and how people can participate. Links: •<a href="http://SecularSolstice.com" rel="nofollow" target="_blank">SecularSolstice.com</a>: <a href="http://secularsolstice.com/" rel="noopener nofollow" target="_blank">http://secularsolstice.com/</a> •Brighter Than Today soundtrack: <a href="https://humanistculture.bandcamp.com/album/brighter-than-today-a-secular-solstice" rel="noopener nofollow" target="_blank">https://humanistculture.bandcamp.com/album/brighter-than-today-a-secular-solstice</a> •Anthony Magnabosco: <a href="https://www.youtube.com/user/magnabosco210" rel="noopener nofollow" target="_blank">https://www.youtube.com/user/magnabosco210</a> •ReasonCon3: <a href="http://reasonnc.com/" rel="noopener nofollow" target="_blank">http://reasonnc.com/</a> •The Biskeptical Podcast: <a href="http://www.spreaker.com/show/the-biskeptical-podcast" rel="noopener nofollow" target="_blank">http://www.spreaker.com/show/the-biskeptical-podcast</a> •Dream Youth: <a href="http://dreamyouth.bandcamp.com" rel="noopener nofollow" target="_blank">http://dreamyouth.bandcamp.com</a> •Asher Silberman: <a href="http://www.ashersilberman.com/" rel="noopener nofollow" target="_blank">http://www.ashersilberman.com/</a> •My Twitter: <a href="http://twitter.com/tmamone" rel="noopener nofollow" target="_blank">http://twitter.com/tmamone</a> •Bi Any Means on Twitter: <a href="http://twitter.com/bianymeanspod" rel="noopener nofollow" target="_blank">http://twitter.com/bianymeanspod</a> •Bi Any Means on Facebook: <a href="http://www.facebook.com/bianymeanspodcast" rel="noopener nofollow" target="_blank">http://www.facebook.com/bianymeanspodcast</a> •The Bi Any Means/Biskeptical Podcast Fans Facebook group: <a href="https://www.facebook.com/groups/179617892470906/" rel="noopener nofollow" target="_blank">https://www.facebook.com/groups/179617892470906/</a> •Email: <a href="mailto:bianymeanspodcast@gmail.com">bianymeanspodcast@gmail.com</a> •Listener line: 410-690-3558 •My Patreon page: <a href="http://www.patreon.com/tmamone" rel="noopener nofollow" target="_blank">http://www.patreon.com/tmamone</a> •Bi Any Means Blog: <a href="http://freethoughtblogs.com/bianymeans/" rel="noopener nofollow" target="_blank">http://freethoughtblogs.com/bianymeans/</a>

Bi Any Means Podcast #77: Secular Solstice with Raymond Arnold

This week I speak with Raymond Arnold, singer/songwriter and author of the popular Winter Solstice service being used around the country. You'll also hear one of his songs, "Brighter Than Today," and comments from listeners about how they're celebrating the holidays "after God."To learn more about Ray's work with the Winter Solstice serve and to hear his music, visit www.humanistculture.com.To learn more about Life After God, visit www.lifeaftergod.org

This week I speak with Raymond Arnold, singer/songwriter and author of the popular Winter Solstice service being used around the country. You'll also hear one of his songs, "Brighter Than Today," and comments from listeners about how they're celebrating the holidays "after God." To learn more about Ray's work with the Winter Solstice serve and to hear his music, visit <a href="http://www.humanistculture.com" rel="noopener nofollow" target="_blank">www.humanistculture.com</a>. To learn more about Life After God, visit <a href="http://www.lifeaftergod.org" rel="noopener nofollow" target="_blank">www.lifeaftergod.org</a>

017 - Winter Solstice with Raymond Arnold