Stephen Merity's guest appearances

Stephen Merity

The TWIML AI Podcast (formerly This Week in Machine Learning & Artificial Intelligence)

Underrated ML

Naila Murray

Sebastian Ruder

Scott Clark

Celeste Kidd

Dennis Mortensen

Rob Walker

Sanmi Koyejo

Chelsea Finn

Dario Amodei

Jianxiong Xiao

Kate Darling

Jason Holmberg

Lotte Bransen

Christopher Berner

David Spiegelhalter

OwlTail

We conclude season one of Underrated ML by having Stephen Merity on as our guest. Stephen has worked at various institutions such as MetaMind and Salesforce ohana, Google Sydney, Freelancer.com, the Schwa Lab at the University of Sydney, the team at Grok Learning, the non-profit Common Crawl, and IACS @ Harvard. He also holds a Bachelor of Information Technology from the University of Sydney and a Master of Science in Computational Science and Engineering from Harvard University.In this weeks episode we talk about the current influences of hardware in the field of Deep Learning research, baseline models, strongly typed RNNs and Alan Turings paper on the chemical basis of morphogenesis. Underrated ML Twitter: https://twitter.com/underrated_mlStephen Merity Twitter: https://twitter.com/SmerityPlease let us know who you thought presented the most underrated paper in the form below: https://forms.gle/97MgHvTkXgdB41TC8Links to the papers:“The Chemical Basis of Morphogenesis” - https://www.dna.caltech.edu/courses/cs191/paperscs191/turing.pdf"Strongly-Typed Recurrent Neural Networks” - https://arxiv.org/abs/1602.02218"Quasi-Recurrent Neural Networks" - https://arxiv.org/abs/1611.01576"An Analysis of Neural Language Modelling at Multiple Scales" - https://arxiv.org/abs/1803.08240Additional Links:Aleatory architecture / hysteresis: Why Birds Are The World's Best EngineersNear decomposability: Near decomposability and the speed of evolution / The Architecture of ComplexityGoogle's All Our N-gram are Belong to You from 2006

We conclude season one of Underrated ML by having Stephen Merity on as our guest. Stephen has worked at various institutions such as MetaMind and Salesforce ohana, Google Sydney, <a href="http://Freelancer.com" rel="nofollow" target="_blank">Freelancer.com</a>, the Schwa Lab at the University of Sydney, the team at Grok Learning, the non-profit Common Crawl, and IACS @ Harvard. He also holds a Bachelor of Information Technology from the University of Sydney and a Master of Science in Computational Science and Engineering from Harvard University. In this weeks episode we talk about the current influences of hardware in the field of Deep Learning research, baseline models, strongly typed RNNs and Alan Turings paper on the chemical basis of morphogenesis. Underrated ML Twitter: <a href="https://twitter.com/underrated_ml" rel="nofollow" target="_blank">https://twitter.com/underrated_ml</a> Stephen Merity Twitter: <a href="https://twitter.com/Smerity" rel="nofollow" target="_blank">https://twitter.com/Smerity</a> Please let us know who you thought presented the most underrated paper in the form below: <a href="https://forms.gle/97MgHvTkXgdB41TC8" rel="nofollow" target="_blank">https://forms.gle/97MgHvTkXgdB41TC8</a> Links to the papers:“The Chemical Basis of Morphogenesis” - <a href="https://www.dna.caltech.edu/courses/cs191/paperscs191/turing.pdf" rel="nofollow" target="_blank">https://www.dna.caltech.edu/courses/cs191/paperscs191/turing.pdf</a>"Strongly-Typed Recurrent Neural Networks” - <a href="https://arxiv.org/abs/1602.02218" rel="nofollow" target="_blank">https://arxiv.org/abs/1602.02218</a>"Quasi-Recurrent Neural Networks" - <a href="https://arxiv.org/abs/1611.01576" rel="nofollow" target="_blank">https://arxiv.org/abs/1611.01576</a> "An Analysis of Neural Language Modelling at Multiple Scales" - <a href="https://arxiv.org/abs/1803.08240" rel="nofollow" target="_blank">https://arxiv.org/abs/1803.08240</a>Additional Links:<ul><li>Aleatory architecture / hysteresis: <a href="https://www.nytimes.com/2020/03/17/science/why-birds-are-the-worlds-best-engineers.html?algo=identity&amp;fellback=false&amp;imp_id=370705999&amp;imp_id=469383815&amp;action=click&amp;module=Science%20%20Technology&amp;pgtype=Homepage" rel="nofollow" target="_blank">Why Birds Are The World's Best Engineers</a></li><li>Near decomposability: <a href="https://academic.oup.com/icc/article-abstract/11/3/587/1044108?redirectedFrom=fulltext" rel="nofollow" target="_blank">Near decomposability and the speed of evolution</a> / <a href="http://www2.econ.iastate.edu/tesfatsi/ArchitectureOfComplexity.HSimon1962.pdf" rel="nofollow" target="_blank">The Architecture of Complexity</a></li><li>Google's <a href="https://ai.googleblog.com/2006/08/all-our-n-gram-are-belong-to-you.html" rel="nofollow" target="_blank">All Our N-gram are Belong to You</a> from 2006</li></ul>

Stephen Merity - Strongly typed RNNs and morphogenesis

Today we’re joined by Stephen Merity, an independent researcher focused on NLP and Deep Learning. In our conversation, we discuss Stephens latest paper, Single Headed Attention RNN: Stop Thinking With Your Head, detailing his primary motivations behind the paper, the decision to use SHA-RNNs for this research, how he built and trained the model, his approach to benchmarking, and finally his goals for the research in the broader research community.

Single Headed Attention RNN: Stop Thinking With Your Head with Stephen Merity - #325