Roshan Rao's guest appearances

Roshan Rao

NLP Highlights

Sebastian Ruder

Ines Montani

Noah Smith

Michael Collins

Brendan O'Connor

Victor Sanh

Sam Bowman

Nathan Schneider

Matt Gardner

Chris Dyer

John Hewitt

Dan Roth

Leon Derczynski

Zhou Yu

Jon Clark

OwlTail

Neil Thomas

For this episode, we chatted with Neil Thomas and Roshan Rao about modeling protein sequences and evaluating transfer learning methods for a set of five protein modeling tasks. Learning representations using self-supervised pretaining objectives has shown promising results in transferring to downstream tasks in protein sequence modeling, just like it has in NLP. We started off by discussing the similarities and differences between language and protein sequence data, and how the contextual embedding techniques are applicable also to protein sequences. Neil and Roshan then described a set of five benchmark tasks to assess the quality of protein embeddings (TAPE), particularly in terms of how well they capture the structural, functional, and evolutionary aspects of proteins. The results from the experiments they ran with various model architectures indicated that there was not a single best performing model across all tasks, and that there is a lot of room for future work in protein sequence modeling.Neil Thomas and Roshan Rao are PhD students at UC Berkeley.Paper: https://www.biorxiv.org/content/10.1101/676825v1Blog post: https://bair.berkeley.edu/blog/2019/11/04/proteins/

For this episode, we chatted with Neil Thomas and Roshan Rao about modeling protein sequences and evaluating transfer learning methods for a set of five protein modeling tasks. Learning representations using self-supervised pretaining objectives has shown promising results in transferring to downstream tasks in protein sequence modeling, just like it has in NLP. We started off by discussing the similarities and differences between language and protein sequence data, and how the contextual embedding techniques are applicable also to protein sequences. Neil and Roshan then described a set of five benchmark tasks to assess the quality of protein embeddings (TAPE), particularly in terms of how well they capture the structural, functional, and evolutionary aspects of proteins. The results from the experiments they ran with various model architectures indicated that there was not a single best performing model across all tasks, and that there is a lot of room for future work in protein sequence modeling. Neil Thomas and Roshan Rao are PhD students at UC Berkeley. Paper: <a href="https://www.biorxiv.org/content/10.1101/676825v1" rel="nofollow" target="_blank">https://www.biorxiv.org/content/10.1101/676825v1</a> Blog post: <a href="https://bair.berkeley.edu/blog/2019/11/04/proteins/" rel="nofollow" target="_blank">https://bair.berkeley.edu/blog/2019/11/04/proteins/</a>

99 - Evaluating Protein Transfer Learning, With Roshan Rao And Neil Thomas