Nelson Elhage's guest appearances

Nelson Elhage

The Accidental Engineer

The Sourcegraph Podcast

The Gradient: Perspectives on AI

Cirrus Labs

Fedor Korotkov

Vegan Donuts

Brian Douglas

Minnie Ingersoll

Josh Evans

Matt Holt

Cassidy Williams

Jose Moreno

Kyle Shevlin

David Cramer

Kelsey Hightower

Jonathan Carter

Peter Cooper

Josh Berkus

OwlTail

Catherine Olsson

Anthropic

In episode 40 of The Gradient Podcast, Andrey Kurenkov speaks to Catherine Olsson and Nelson Elhage.Catherine and Nelson are both members of technical staff at Anthropic, which is an AI safety and research company that’s working to build reliable, interpretable, and steerable AI systems. Catherine and Nelson’s focus is on interpretability, and we will discuss several of their recent works in this interview. Follow The Gradient on TwitterOutline:(00:00) Intro(01:10) Catherine’s Path into AI(03:25) Nelson’s Path into AI(05:23) Overview of Anthropic(08:21) Mechanistic Interpretability(15:15) Transformer Circuits (21:30) Toy Transformer(27:25) Induction Heads(31:00) In-Context Learning(35:10) Evidence for Induction Heads Enabling In-Context Learning(39:30) What’s Next(43:10) Replicating Results(46:00) OutroLinks:AnthropicZoom In: An Introduction to CircuitsMechanistic Interpretability, Variables, and the Importance of Interpretable BasesA Mathematical Framework for Transformer CircuitsIn-context Learning and Induction Heads PySvelte Get full access to The Gradient at thegradientpub.substack.com/subscribe

In episode 40 of The Gradient Podcast, <a href="https://www.andreykurenkov.com/" rel="noopener noreferrer nofollow" target="_blank">Andrey Kurenkov</a> speaks to Catherine Olsson and Nelson Elhage. Catherine and Nelson are both members of technical staff at Anthropic, which is an AI safety and research company that’s working to build reliable, interpretable, and steerable AI systems. Catherine and Nelson’s focus is on interpretability, and we will discuss several of their recent works in this interview.  Follow The Gradient on <a href="https://twitter.com/gradientpub" rel="noopener noreferrer nofollow" target="_blank">Twitter</a> Outline: (00:00) Intro (01:10) Catherine’s Path into AI (03:25) Nelson’s Path into AI (05:23) Overview of Anthropic (08:21) Mechanistic Interpretability (15:15) Transformer Circuits  (21:30) Toy Transformer (27:25) Induction Heads (31:00) In-Context Learning (35:10) Evidence for Induction Heads Enabling In-Context Learning (39:30) What’s Next (43:10) Replicating Results (46:00) Outro Links: <a href="https://www.anthropic.com/" rel="noopener noreferrer nofollow" target="_blank">Anthropic</a> <a href="https://distill.pub/2020/circuits/zoom-in/" rel="noopener noreferrer nofollow" target="_blank">Zoom In: An Introduction to Circuits</a> <a href="https://transformer-circuits.pub/2022/mech-interp-essay/index.html" rel="noopener noreferrer nofollow" target="_blank">Mechanistic Interpretability, Variables, and the Importance of Interpretable Bases</a> <a href="https://transformer-circuits.pub/2021/framework/index.html" rel="noopener noreferrer nofollow" target="_blank">A Mathematical Framework for Transformer Circuits</a> <a href="https://transformer-circuits.pub/2022/in-context-learning-and-induction-heads/index.html" rel="noopener noreferrer nofollow" target="_blank">In-context Learning and Induction Heads</a> <a href="https://github.com/anthropics/PySvelte" rel="noopener noreferrer nofollow" target="_blank">PySvelte</a> Get full access to The Gradient at <a href="https://thegradientpub.substack.com/subscribe?utm_medium=podcast&amp;utm_campaign=CTA_4" rel="nofollow" target="_blank">thegradientpub.substack.com/subscribe</a>

Catherine Olsson and Nelson Elhage: Anthropic, Understanding Transformers

Why is a systems engineering mindset essential for a scaling startup? In this episode of the Sourcegraph Podcast, Nelson Elhage, creator of the open source code search engine Livegrep, co-creator of the Ruby type checker Sorbet, and Member of Technical Staff at Anthropic, joins Beyang Liu, co-founder and CTO of Sourcegraph, to discuss how Rust is changing the security landscape, explain why Patrick McKenzie, better known as patio11, called his live code search tool “miraculous,” and dive deep into the weeds on the differences between trigram- and suffix-array-based search systems. Along the way, Elhage explains why developer productivity is nonlinear and why investing in developer experience should be axiomatic.Show notes &amp; transcript: https://about.sourcegraph.com/podcast/nelson-elhage/Sourcegraph: about.sourcegraph.com

Why is a systems engineering mindset essential for a scaling startup? In this episode of the Sourcegraph Podcast, Nelson Elhage, creator of the open source code search engine Livegrep, co-creator of the Ruby type checker Sorbet, and Member of Technical Staff at Anthropic, joins Beyang Liu, co-founder and CTO of Sourcegraph, to discuss how Rust is changing the security landscape, explain why Patrick McKenzie, better known as patio11, called his live code search tool “miraculous,” and dive deep into the weeds on the differences between trigram- and suffix-array-based search systems. Along the way, Elhage explains why developer productivity is nonlinear and why investing in developer experience should be axiomatic. Show notes &amp; transcript: <a href="https://about.sourcegraph.com/podcast/nelson-elhage/" rel="nofollow" target="_blank">https://about.sourcegraph.com/podcast/nelson-elhage/</a> Sourcegraph: <a href="http://about.sourcegraph.com" rel="nofollow" target="_blank">about.sourcegraph.com</a>

Disassembling and building developer tools, with Nelson Elhage, creator of open source code search engine Livegrep

Nelson Elhage most recently spent 7 years at Stripe, spearheading developer productivity initiatives.

Testing at Scale: Nelson Elhage