Andrew Saxe's guest appearances

Andrew Saxe

Cinema with a Twist

Brain Inspired

Epistimones

Chris Eliasmith

Randy Gallistel

David Rosen

Brian Jay Jones

Ryan Poplin

Blake Richards

Terrence Sejnowski

David Ferrucci

Raia Hadsell

Melanie Mitchell

Danny Rubin

Matt Smith

Alison Gopnik

Jim Henson

Brian Christian

OwlTail

Andrew Saxe is a theoretical neuroscientist and is a Sir Henry Dale Fellow and Joint Group Leader at the Gatsby Computational Neuroscience Unit and Sainsbury Wellcome Centre at UCL. His research focuses on developing a mathematical toolkit for analysing and describing phenomena in neuroscience. Currently, he is working on the theory of deep learning, which uses artificial neural network models, and aims to apply this to explain how learning is instantiated in neural networks in the brain.  TIMESTAMPS  (00:38) – Nature of intelligence (10:23) – Theoretical neuroscience (11:31) – What attracted Andrew to theoretical neuroscience (12:16) – Best route to get into theoretical neuroscience (16:55) – Collaborating with experimental neuroscientists (25:34) – Artificial vs biological neural networks (34:06) – What deep learning has taught us about the brain (37:32) – Linear neural networks (43:34) – Grand unified theory of neural networks (45:35) – Incorporating biological aspects into neural networks (48:53) – Artificial general intelligence (51:13) – Is maths invented or discovered? (52:29) – Advice for young scientists

Andrew Saxe is a theoretical neuroscientist and is a Sir Henry Dale Fellow and Joint Group Leader at the Gatsby Computational Neuroscience Unit and Sainsbury Wellcome Centre at UCL. His research focuses on developing a mathematical toolkit for analysing and describing phenomena in neuroscience. Currently, he is working on the theory of deep learning, which uses artificial neural network models, and aims to apply this to explain how learning is instantiated in neural networks in the brain. TIMESTAMPS (00:38) – Nature of intelligence (10:23) – Theoretical neuroscience (11:31) – What attracted Andrew to theoretical neuroscience (12:16) – Best route to get into theoretical neuroscience (16:55) – Collaborating with experimental neuroscientists (25:34) – Artificial vs biological neural networks (34:06) – What deep learning has taught us about the brain (37:32) – Linear neural networks (43:34) – Grand unified theory of neural networks (45:35) – Incorporating biological aspects into neural networks (48:53) – Artificial general intelligence (51:13) – Is maths invented or discovered? (52:29) – Advice for young scientists

#3 – Andrew Saxe: Deep learning and the brain

Support the Podcast Andrew and I discuss his work exploring how various facets of deep networks contribute to their function, i.e. deep network theory. We talk about what he’s learned by studying linear deep networks and asking how depth and initial weights affect learning dynamics, when replay is appropriate (and when it’s not), how semantics develop, and what it all might tell us about deep learning in brains. Show notes: Visit Andrew's website. The papers we discuss or mention: Are Efficient Deep Representations Learnable?A theory of memory replay and generalization performance in neural networks.A mathematical theory of semantic development in deep neural networks.A good talk: High-Dimensional Dynamics Of Generalization Errors.  A few recommended texts to dive deeper: Introduction To The Theory Of Neural Computation.Statistical Mechanics of Learning.Theoretical Neuroscience.

Support the Podcast Andrew and I discuss his work exploring how various facets of deep networks contribute to their function, i.e. deep network theory. We talk about what he’s learned by studying linear deep networks and asking how depth and initial weights affect learning dynamics, when replay is appropriate (and when it’s not), how semantics develop, and what it all might tell us about deep learning in brains. Show notes: <ul><li>Visit Andrew's <a href="http://decisions.psy.ox.ac.uk/people/saxe_site/" rel="nofollow" target="_blank">website</a>. </li><li>The papers we discuss or mention: <ul><li><a href="https://arxiv.org/abs/1807.06399" rel="nofollow" target="_blank">Are Efficient Deep Representations Learnable?</a></li><li><a href="http://decisions.psy.ox.ac.uk/people/saxe_site/papers/Saxe,%20Advani%20-%202018%20-%20A%20theory%20of%20memory%20replay%20and%20generalization%20performance%20in%20neural%20networks.pdf" rel="nofollow" target="_blank">A theory of memory replay and generalization performance in neural networks.</a></li><li><a href="http://decisions.psy.ox.ac.uk/people/saxe_site/papers/Saxe,%20McClelland,%20Ganguli%20-%202018%20-%20A%20mathematical%20theory%20of%20semantic%20development%20in%20deep%20neural%20networks.pdf" rel="nofollow" target="_blank">A mathematical theory of semantic development in deep neural networks.</a></li></ul></li><li>A good talk: <ul><li><a href="https://www.youtube.com/watch?v=DuynTHJ8y8Y&amp;list=PLxQIghRi8KAZnIE1iZe4KeIO4zWTSocdk&amp;index=5&amp;t=0s" rel="nofollow" target="_blank">High-Dimensional Dynamics Of Generalization Errors</a>. </li></ul></li></ul> A few recommended texts to dive deeper: <ul><li><a href="https://www.amazon.com/gp/product/0201515601/ref=as_li_tl?ie=UTF8&amp;tag=pmiddlebroo09-20&amp;camp=1789&amp;creative=9325&amp;linkCode=as2&amp;creativeASIN=0201515601&amp;linkId=c3782edc9d33d69bce7584dd87487517" rel="nofollow" target="_blank">Introduction To The Theory Of Neural Computation</a>.</li><li><a href="https://www.amazon.com/gp/product/0521773075/ref=as_li_tl?ie=UTF8&amp;camp=1789&amp;creative=9325&amp;creativeASIN=0521773075&amp;linkCode=as2&amp;tag=pmiddlebroo09-20&amp;linkId=7f35d760f4e4bf11747c6183f366cb48" rel="nofollow" target="_blank">Statistical Mechanics of Learning.</a></li><li><a href="https://www.amazon.com/gp/product/0262541858/ref=as_li_tl?ie=UTF8&amp;tag=pmiddlebroo09-20&amp;camp=1789&amp;creative=9325&amp;linkCode=as2&amp;creativeASIN=0262541858&amp;linkId=aba4413f84a056ad9b800a74b2116796" rel="nofollow" target="_blank">Theoretical Neuroscience</a>.</li></ul>

BI 052 Andrew Saxe: Deep Learning Theory

Interview with veteran film/television production manager, Andrew Saxe. Original air date: May 4, 2016For more information about Cinema with a Twist, visit: facebook.com/cinemawithatwist

Interview with veteran film/television production manager, Andrew Saxe. Original air date: May 4, 2016 For more information about Cinema with a Twist, visit: <a href="http://facebook.com/cinemawithatwist" rel="nofollow" target="_blank">facebook.com/cinemawithatwist</a>

Episode 25: Andrew Saxe, Production Manager