Mayank Bansal's guest appearances

Mayank Bansal

Cloud Engineering Archives - Software Engineering Daily

Software Daily

IRadioLive Podcasting Platform (www.i-radiolive.com)

Anurag Gupta

Ellen Chisa

Eric Anderson

Adrian Cockcroft

Brendan Burns

Alex Ross

Ville Aikas

Alexandr Wang

Julien Lemoine

David Rice

Ryan Nystrom

Brian Lovin

Abby Fuller

Mark Carter

Guillermo Rauch

OwlTail

Min Cai

Upcoming events: A Conversation with Haseeb Qureshi at Cloudflare on April 3, 2019 FindCollabs Hackathon at App Academy on April 6, 2019 Google’s Borg system is a cluster manager that powers the applications running across Google’s massive infrastructure. Borg provided inspiration for open source tools like Apache Mesos and Kubernetes. Over the last decade, some of the largest new technology companies have built their own systems that fulfill the roles of cluster management and resource scheduling. Netflix, Twitter, and Facebook have all spoken about their internal projects to make distributed systems resource allocation more economical. These companies find themselves continually reinventing scheduling and orchestration, with inspiration from Google Borg and their own internal experiences running large numbers of containers and virtual machines. Uber’s engineering team has built a cluster scheduler called Peloton. Peloton is based on Apache Mesos, and is architected to handle a wide range of workloads: data science jobs like Hadoop MapReduce; long running services such as a ridesharing marketplace service; monitoring daemons such as Uber’s M3 collector; and database services such as MySQL. Min Cai and Mayank Bansal are engineers at Uber who work on Peloton. When they set out to create Peloton, they looked at the existing schedulers in the ecosystem, including Kubernetes, Mesos, Hadoop’s YARN system, and Borg itself.  Both Min and Mayank join the show today to give a brief history of distributed systems schedulers and discuss their work on Peloton. They have been working in the world of distributed systems schedulers for many years–including experiences building core Hadoop infrastructure and virtual machine schedulers at VMware. The post Peloton: Uber’s Cluster Scheduler with Min Cai and Mayank Bansal appeared first on Software Engineering Daily.

Upcoming events: <a href="https://softwareengineeringdaily.com/meetup" rel="nofollow" target="_blank">A Conversation with Haseeb Qureshi at Cloudflare on April 3, 2019</a> <a href="https://softwareengineeringdaily.com/hackathon" rel="nofollow" target="_blank">FindCollabs Hackathon at App Academy on April 6, 2019</a> <a href="https://softwareengineeringdaily.com/2018/04/27/google-cluster-evolution-with-brian-grant/" rel="nofollow" target="_blank">Google’s Borg system</a> is a cluster manager that powers the applications running across Google’s massive infrastructure. Borg provided inspiration for open source tools like Apache Mesos and Kubernetes. Over the last decade, some of the largest new technology companies have built their own systems that fulfill the roles of cluster management and resource scheduling. <a href="https://softwareengineeringdaily.com/2016/10/12/netflix-scheduling-with-sharma-podila/" rel="nofollow" target="_blank">Netflix</a>, Twitter, and <a href="https://www.youtube.com/watch?v=C_WuUgTqgOc" rel="nofollow" target="_blank">Facebook</a> have all spoken about their internal projects to make distributed systems resource allocation more economical. These companies find themselves continually reinventing scheduling and orchestration, with inspiration from Google Borg and their own internal experiences running large numbers of containers and virtual machines. Uber’s engineering team has built a cluster scheduler called Peloton. Peloton is based on Apache Mesos, and is architected to handle a wide range of workloads: data science jobs like Hadoop MapReduce; long running services such as a ridesharing marketplace service; monitoring daemons such as <a href="https://softwareengineeringdaily.com/2019/02/12/ubers-monitoring-platform-with-rob-skillington/" rel="nofollow" target="_blank">Uber’s M3 collector</a>; and database services such as MySQL. Min Cai and Mayank Bansal are engineers at Uber who work on Peloton. When they set out to create Peloton, they looked at the existing schedulers in the ecosystem, including<a href="https://softwareengineeringdaily.com/2016/04/19/googles-container-management-brendan-burns/" rel="nofollow" target="_blank"> Kubernetes, Mesos, Hadoop’s YARN system, and Borg itself</a>. Both Min and Mayank join the show today to give a brief history of distributed systems schedulers and discuss their work on Peloton. They have been working in the world of distributed systems schedulers for many years–including experiences building core Hadoop infrastructure and virtual machine schedulers at VMware. The post <a href="https://softwareengineeringdaily.com/2019/03/28/peloton-ubers-cluster-scheduler-with-min-cai-and-mayank-bansal/" rel="nofollow" target="_blank">Peloton: Uber’s Cluster Scheduler with Min Cai and Mayank Bansal</a> appeared first on <a href="https://softwareengineeringdaily.com" rel="nofollow" target="_blank">Software Engineering Daily</a>.

Peloton: Uber’s Cluster Scheduler with Min Cai and Mayank Bansal