Josh Hickman's guest appearances

Josh Hickman

Catalog of Interviews and Bits

The Crude Life 

The Real Enneagram

The Nonlinear Library

Brock Pierce

Joryn Jenkins

Robert Hardgrave

Julie Jason

Brenda Wiebe

Crispin Boyer

Vince Aiello

Ed Schultz

Nicole Dieker

Chris Papst

Shari Zucker

Archie Scott

Mark Alyn

Kathy Sparrow

Alan Murphy

OwlTail

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: A Rephrasing Of and Footnote To An Embedded Agency Proposal, published by Josh Hickman on March 9, 2022 on The AI Alignment Forum.I wanted to clarify some comments made in this post, which proposed a way to resolve the issues brought up in the Action Counterfactuals subsection of the Embedded Agency paper. I noticed that my ideal edits would end up rewriting the post almost completely (it wasn't particularly clear), and I wanted to more coherently lay out the thinking.The ProblemThe problem is that we'd like agents to consider action counterfactuals, and, perhaps, find a proof that an action leads to the highest reward before taking it. The formulation in the paper leads to some Lobian issues, where agents choose actions by searching over proofs for statements similar to:[The paper actually has a much cleaner phrasing of the statements it searches for proofs for, to make the proof issue much clearer.]This is a rather substantial bummer, because something like that would be great to rely on. It'll find the proofs for what you'd like, but it'll also find proofs for nonsense. The Embedded Agency paper is a great resource for this, but you should also work out why this might happen in your head.The ConfusionOne potential intuition for why this problem exists is the following:The agent who is taking this argmax always chooses the highest reward thing. If I know the rewards for all actions but the last X, well, choosing X means it was the result of the argmax. It's right there in the source code / function definition. So W must have been the biggest! Therefore, X is the result of the argmax being computed, because even if we don't know W, I know that the agent choosing X means W is bigger than all the other W' values. It must be -- the choice was coming from an agent doing an argmax!This might be a confusing phrasing. The ideal way to explain this might be setting up the 5-10 problem in a theorem prover and just showing people it'll find the bad proof. But I think the narrative version captures something coherent.The confusion we should have is that this is not how we, humans, engage in action counterfactuals! Not even close. Normally that's not an issue, but let's pay more attention to see if the difference is substantive.In the original post, I use the example of a professor considering whether to slash the tires on his car. He ponders what it would imply, if he chose to do that. It would imply he must desperately not want the ability to travel, plausibly. It might imply he's driven into a river and needs access to the air within them for safety, like he's seen in movies. He's a sensible person -- him making the choice must mean there's a very good reason he did so! Much better than the standard "I'm just trying to get to work" reasons to not slash the tires.But the professor, running his weaker, informal decision making process, does not slash his tires. You don't want to know the rewards implied by choosing to take an action. Although the professor has no Lobian issues in his informal decision-making, he views the future differently: he asks what would happen if he was the same type of person who just made the choice, excluding any consideration of why he might be making the decision, and considers what might happen to that imaginary person.The IntuitionThe intuition I was using was: why can't the agents do something similar? Create a new agent that "teleports them forward in time" by just precommiting to making the decision, and seeing what results? This matches our informal understanding of the phrase "action counterfactual" much better, I think, and because we're creating a new agent (one with this one decision pre-set into it), it's possible the self-reference issues go away.Does it mechanically work to resolve the issue?It does in the specific exam...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: A Rephrasing Of and Footnote To An Embedded Agency Proposal, published by Josh Hickman on March 9, 2022 on The AI Alignment Forum. I wanted to clarify some comments made in this post, which proposed a way to resolve the issues brought up in the Action Counterfactuals subsection of the Embedded Agency paper. I noticed that my ideal edits would end up rewriting the post almost completely (it wasn't particularly clear), and I wanted to more coherently lay out the thinking. The Problem The problem is that we'd like agents to consider action counterfactuals, and, perhaps, find a proof that an action leads to the highest reward before taking it. The formulation in the paper leads to some Lobian issues, where agents choose actions by searching over proofs for statements similar to: [The paper actually has a much cleaner phrasing of the statements it searches for proofs for, to make the proof issue much clearer.] This is a rather substantial bummer, because something like that would be great to rely on. It'll find the proofs for what you'd like, but it'll also find proofs for nonsense. The Embedded Agency paper is a great resource for this, but you should also work out why this might happen in your head. The Confusion One potential intuition for why this problem exists is the following: The agent who is taking this argmax always chooses the highest reward thing. If I know the rewards for all actions but the last X, well, choosing X means it was the result of the argmax. It's right there in the source code / function definition. So W must have been the biggest! Therefore, X is the result of the argmax being computed, because even if we don't know W, I know that the agent choosing X means W is bigger than all the other W' values. It must be -- the choice was coming from an agent doing an argmax! This might be a confusing phrasing. The ideal way to explain this might be setting up the 5-10 problem in a theorem prover and just showing people it'll find the bad proof. But I think the narrative version captures something coherent. The confusion we should have is that this is not how we, humans, engage in action counterfactuals! Not even close. Normally that's not an issue, but let's pay more attention to see if the difference is substantive. In the original post, I use the example of a professor considering whether to slash the tires on his car. He ponders what it would imply, if he chose to do that. It would imply he must desperately not want the ability to travel, plausibly. It might imply he's driven into a river and needs access to the air within them for safety, like he's seen in movies. He's a sensible person -- him making the choice must mean there's a very good reason he did so! Much better than the standard "I'm just trying to get to work" reasons to not slash the tires. But the professor, running his weaker, informal decision making process, does not slash his tires. You don't want to know the rewards implied by choosing to take an action. Although the professor has no Lobian issues in his informal decision-making, he views the future differently: he asks what would happen if he was the same type of person who just made the choice, excluding any consideration of why he might be making the decision, and considers what might happen to that imaginary person. The Intuition The intuition I was using was: why can't the agents do something similar? Create a new agent that "teleports them forward in time" by just precommiting to making the decision, and seeing what results? This matches our informal understanding of the phrase "action counterfactual" much better, I think, and because we're creating a new agent (one with this one decision pre-set into it), it's possible the self-reference issues go away. Does it mechanically work to resolve the issue? It does in the specific exam...

AF - A Rephrasing Of and Footnote To An Embedded Agency Proposal by Josh Hickman

Josh Hickman, American Energy Partners, gives an overview of American Energy Partners as well as his professional background.   The paradigm shift in energy is discussed from the well site to the office to financing.   Hickman gives an update from the Marcellus and Utica shale plays, as well as his thoughts on working in Texas, Colorado and the Bakken.  Land agents, perception of fossil fuels, agriculture and clean energy are also discussed.   Couple interesting topics from the interview also include technologies role in supply and demand, the fact that pipelines are booming and where the cash flow plays are. The Crude Life Podcast can be heard Monday through Thursday with a Week in Review every Friday. Subscribe on iTunes Subscribe on iHeartRadio Subscribe on RadioRepublic Subscribe on ListenNotes Spread the word. Support the industry. Share the energy. Follow on YouTube https://www.youtube.com/channel/UCtVimPyUygnwakoQ9swHXNA Follow on Facebook https://www.facebook.com/thecrudelife/ Follow on LinkedIn https://www.linkedin.com/company/the-crude-life Follow on Twitter https://twitter.com/RealCrudeLife  #post-7249 .CPlase_panel {display:none;}

Josh Hickman, American Energy Partners, gives an overview of American Energy Partners as well as his professional background. The paradigm shift in energy is discussed from the well site to the office to financing. Hickman gives an update from the Marcellus and Utica shale plays, as well as his thoughts on working in Texas, Colorado and the Bakken. Land agents, perception of fossil fuels, agriculture and clean energy are also discussed. Couple interesting topics from the interview also include technologies role in supply and demand, the fact that pipelines are booming and where the cash flow plays are. The Crude Life Podcast can be heard Monday through Thursday with a Week in Review every Friday. <a href="https://podcasts.apple.com/us/podcast/the-crude-life-media-network/id893728790" rel="nofollow" target="_blank">Subscribe on iTunes</a> <a href="https://www.iheart.com/podcast/256-the-crude-life-media-netwo-30961495/" rel="nofollow" target="_blank">Subscribe on iHeartRadio</a> <a href="https://radiopublic.com/the-crude-life-WdvnaR/episodes" rel="nofollow" target="_blank">Subscribe on RadioRepublic</a> <a href="https://www.listennotes.com/podcasts/the-crude-life-the-crude-life-IXbwiaUyZGY/" rel="nofollow" target="_blank">Subscribe on ListenNotes</a> Spread the word. Support the industry. Share the energy. Follow on YouTube <a href="https://www.youtube.com/channel/UCtVimPyUygnwakoQ9swHXNA" rel="nofollow" target="_blank">https://www.youtube.com/channel/UCtVimPyUygnwakoQ9swHXNA</a> Follow on Facebook <a href="https://www.facebook.com/thecrudelife" rel="nofollow" target="_blank">https://www.facebook.com/thecrudelife</a>/ Follow on LinkedIn <a href="https://www.linkedin.com/company/the-crude-life" rel="nofollow" target="_blank">https://www.linkedin.com/company/the-crude-life</a> Follow on Twitter <a href="https://twitter.com/RealCrudeLife" rel="nofollow" target="_blank">https://twitter.com/RealCrudeLife</a> #post-7249 .CPlase_panel {display:none;}

The Crude Life Interview: Josh Hickman, American Energy Partners

Josh Hickman author of FIVE SLICES OF FEAR for Halloween- based on comic novels