Victor Sanh's guest appearances

Victor Sanh

NLP Highlights

Chai Time Data Science 

Ines Montani

Thomas Wolf

Sara Hooker

Judy Gichoya

Rachel Thomas

Noah Smith

Anthony Goldbloom

Andrew Shaw

Richard Craib

Michael Collins

John Miller

Ian Ozsvald

Michael Kennedy

Bryan Catanzaro

Jose Fernandez

OwlTail

In this episode we talked with Victor Sanh and Thomas Wolf from HuggingFace about model distillation, and DistilBERT as one example of distillation. The idea behind model distillation is compressing a large model by building a smaller model, with much fewer parameters, that approximates the output distribution of the original model, typically for increased efficiency. We discussed how model distillation was typically done previously, and then focused on the specifics of DistilBERT, including training objective, empirical results, ablations etc. We finally discussed what kinds of information you might lose when doing model distillation.

104 - Model Distillation, with Victor Sanh and Thomas Wolf

Hugging Face

Audio (Podcast Version) available here: https://anchor.fm/chaitimedatascience  Subscribe here to the newsletter: https://tinyletter.com/sanyambhutani  In this episode, Sanyam Bhutani interviews the Victor Sanh, Research Scientist at Hugging Face. In this interview, they talk all about his journey into the field of NLP and Machine Learning.    They also talk about Research in general and Research at Hugging Face along with a discussion on DistilBERT.   Links:   DistilBERT: https://medium.com/huggingface/distilbert-8cf3380435b5  https://arxiv.org/abs/1910.01108 Transformers Repo:  https://github.com/huggingface/transformers https://medium.com/huggingface/distilbert-8cf3380435b5  Follow: Victor Sanh:  https://twitter.com/sanhestpasmoi?lang=en  https://www.linkedin.com/in/victor-sanh/?locale=en_US  Hugging Face:  https://twitter.com/huggingface  https://huggingface.co/  https://medium.com/huggingface  Sanyam Bhutani:  https://twitter.com/bhutanisanyam1  Blog: sanyambhutani.com About:  http://chaitimedatascience.com/  A show for Interviews with Practitioners, Kagglers &amp; Researchers and all things Data Science hosted by Sanyam Bhutani.    You can expect weekly episodes every available as Video, Podcast, and blogposts.   If you'd like to support the podcast: https://www.patreon.com/chaitimedatascience  Intro track:  Flow by LiQWYD https://soundcloud.com/liqwyd

Audio (Podcast Version) available here: <a href="https://anchor.fm/chaitimedatascience" rel="nofollow" target="_blank">https://anchor.fm/chaitimedatascience</a> Subscribe here to the newsletter: <a href="https://tinyletter.com/sanyambhutani" rel="nofollow" target="_blank">https://tinyletter.com/sanyambhutani</a> In this episode, Sanyam Bhutani interviews the Victor Sanh, Research Scientist at Hugging Face. In this interview, they talk all about his journey into the field of NLP and Machine Learning.    They also talk about Research in general and Research at Hugging Face along with a discussion on DistilBERT.  Links:   DistilBERT: <a href="https://medium.com/huggingface/distilbert-8cf3380435b5" rel="nofollow" target="_blank">https://medium.com/huggingface/distilbert-8cf3380435b5</a> <a href="https://arxiv.org/abs/1910.01108" rel="nofollow" target="_blank">https://arxiv.org/abs/1910.01108</a> Transformers Repo:  <a href="https://github.com/huggingface/transformers" rel="nofollow" target="_blank">https://github.com/huggingface/transformers</a> <a href="https://medium.com/huggingface/distilbert-8cf3380435b5" rel="nofollow" target="_blank">https://medium.com/huggingface/distilbert-8cf3380435b5</a> Follow: Victor Sanh:  <a href="https://twitter.com/sanhestpasmoi?lang=en" rel="nofollow" target="_blank">https://twitter.com/sanhestpasmoi?lang=en</a> <a href="https://www.linkedin.com/in/victor-sanh/?locale=en_US" rel="nofollow" target="_blank">https://www.linkedin.com/in/victor-sanh/?locale=en_US</a> Hugging Face:  <a href="https://twitter.com/huggingface" rel="nofollow" target="_blank">https://twitter.com/huggingface</a> <a href="https://huggingface.co/" rel="nofollow" target="_blank">https://huggingface.co/</a> <a href="https://medium.com/huggingface" rel="nofollow" target="_blank">https://medium.com/huggingface</a> Sanyam Bhutani:  <a href="https://twitter.com/bhutanisanyam1" rel="nofollow" target="_blank">https://twitter.com/bhutanisanyam1</a> Blog: <a href="http://sanyambhutani.com" rel="nofollow" target="_blank">sanyambhutani.com</a> About:  <a href="http://chaitimedatascience.com/" rel="nofollow" target="_blank">http://chaitimedatascience.com/</a> A show for Interviews with Practitioners, Kagglers &amp; Researchers and all things Data Science hosted by Sanyam Bhutani.    You can expect weekly episodes every available as Video, Podcast, and blogposts.   If you'd like to support the podcast: <a href="https://www.patreon.com/chaitimedatascience" rel="nofollow" target="_blank">https://www.patreon.com/chaitimedatascience</a> Intro track:  Flow by LiQWYD <a href="https://soundcloud.com/liqwyd" rel="nofollow" target="_blank">https://soundcloud.com/liqwyd</a>

DistilBERT | Research at Hugging Face | NLP and Open Source | Interview with Victor Sanh