Kapil Surlaker's guest appearances

Kapil Surlaker

Software Engineering Daily

Data Archives - Software Engineering Daily

Podcast Archives - Software Engineering Daily

Software Daily

Zhenxiao Luo

Carl Steinbach

Alan Murphy

Vinoth Chandar

Sam Shah

Kevin Xu

Wenbin Fang

Kalyan Ramanathan

Joshua Haas

John Whaley

Michael Armbrust

Matthew Rocklin

Allison King

Sujith Nair

David Potere

OwlTail

A large social network needs to develop systems for ingesting, storing, and processing large volumes of data.Data engineering at scale requires multiple engineering teams that are responsible for different areas of the infrastructure.Data needs to be structured coherently in order to minimize the data cleaning process. Machine learning models need to be developed, deployed, and iterated on at scale. Areas of the company which produce data need to be decoupled from the areas of the company which consume data, so that engineers throughout the company can reliably build tools on top of these large data sets.In our previous episodes about LinkedIn, we covered two major components of LinkedIn’s data engineering systems: the Kafka infrastructure and the LinkedIn data platform used by engineers to productively build data applications.Kapil Surlaker is a senior director of engineering at LinkedIn, and he joins the show to discuss the bigger picture of LinkedIn’s data infrastructure. Kapil works with teams across LinkedIn to understand the requirements for the products and internal tools, and translate those requirements into team structures and software platforms that let LinkedIn use data more productively.We discuss a wide range of topics, including engineering management, the modern data platform, and LinkedIn’s adoption of public cloud.Full disclosure: LinkedIn is a sponsor of Software Engineering Daily. 

A large social network needs to develop systems for ingesting, storing, and processing large volumes of data.Data engineering at scale requires multiple engineering teams that are responsible for different areas of the infrastructure.Data needs to be structured coherently in order to minimize the data cleaning process. Machine learning models need to be developed, deployed, and iterated on at scale. Areas of the company which produce data need to be decoupled from the areas of the company which consume data, so that engineers throughout the company can reliably build tools on top of these large data sets.In our previous episodes about LinkedIn, we covered two major components of LinkedIn’s data engineering systems: the Kafka infrastructure and the LinkedIn data platform used by engineers to productively build data applications.Kapil Surlaker is a senior director of engineering at LinkedIn, and he joins the show to discuss the bigger picture of LinkedIn’s data infrastructure. Kapil works with teams across LinkedIn to understand the requirements for the products and internal tools, and translate those requirements into team structures and software platforms that let LinkedIn use data more productively.We discuss a wide range of topics, including engineering management, the modern data platform, and LinkedIn’s adoption of public cloud.Full disclosure: LinkedIn is a sponsor of Software Engineering Daily.

LinkedIn Data Engineering with Kapil Surlaker

A large social network needs to develop systems for ingesting, storing, and processing large volumes of data. Data engineering at scale requires multiple engineering teams that are responsible for different areas of the infrastructure. Data needs to be structured coherently in order to minimize the data cleaning process. Machine learning models need to be developed, deployed, and iterated on at scale. Areas of the company which produce data need to be decoupled from the areas of the company which consume data, so that engineers throughout the company can reliably build tools on top of these large data sets. In our previous episodes about LinkedIn, we covered two major components of LinkedIn’s data engineering systems: the Kafka infrastructure and the LinkedIn data platform used by engineers to productively build data applications. Kapil Surlaker is a senior director of engineering at LinkedIn, and he joins the show to discuss the bigger picture of LinkedIn’s data infrastructure. Kapil works with teams across LinkedIn to understand the requirements for the products and internal tools, and translate those requirements into team structures and software platforms that let LinkedIn use data more productively. We discuss a wide range of topics, including engineering management, the modern data platform, and LinkedIn’s adoption of public cloud. Full disclosure: LinkedIn is a sponsor of Software Engineering Daily.  Sponsorship inquiries: sponsor@softwareengineeringdaily.com Check out our active projects:  We are hiring a head of growth. If you like Software Engineering Daily and consider yourself competent in sales, marketing, and strategy, send me an email: jeff@softwareengineeringdaily.com FindCollabs is a place to build open source software. The SEDaily app for iOS and Android includes all 1000 of our old episodes, as well as related links, greatest hits, and topics. Subscribe for ad-free episodes.  The post LinkedIn Data Engineering with Kapil Surlaker appeared first on Software Engineering Daily.

A large social network needs to develop systems for ingesting, storing, and processing large volumes of data. Data engineering at scale requires multiple engineering teams that are responsible for different areas of the infrastructure. Data needs to be structured coherently in order to minimize the data cleaning process. Machine learning models need to be developed, deployed, and iterated on at scale. Areas of the company which produce data need to be decoupled from the areas of the company which consume data, so that engineers throughout the company can reliably build tools on top of these large data sets. In our previous episodes about LinkedIn, we covered two major components of LinkedIn’s data engineering systems: the Kafka infrastructure and the LinkedIn data platform used by engineers to productively build data applications. Kapil Surlaker is a senior director of engineering at LinkedIn, and he joins the show to discuss the bigger picture of LinkedIn’s data infrastructure. Kapil works with teams across LinkedIn to understand the requirements for the products and internal tools, and translate those requirements into team structures and software platforms that let LinkedIn use data more productively. We discuss a wide range of topics, including engineering management, the modern data platform, and LinkedIn’s adoption of public cloud. Full disclosure: LinkedIn is a sponsor of Software Engineering Daily.  Sponsorship inquiries: <a href="mailto:sponsor@softwareengineeringdaily.com">sponsor@softwareengineeringdaily.com</a> <h2>Check out our active projects:</h2> <ul> <li>We are hiring a <a href="https://www.softwaredaily.com/jobs/5c3a6cf0f7fb250029d873a9" rel="nofollow" target="_blank">head of growth</a>. If you like Software Engineering Daily and consider yourself competent in sales, marketing, and strategy, send me an email: jeff@softwareengineeringdaily.com</li> <li><a href="https://findcollabs.com/" rel="nofollow" target="_blank">FindCollabs</a> is a place to build open source software.</li> <li><a href="https://itunes.apple.com/us/app/software-engineering-daily/id1253734426?mt=8" rel="nofollow" target="_blank">The SEDaily app for iOS</a> and <a href="https://play.google.com/store/apps/details?id=com.koalatea.sedaily" rel="nofollow" target="_blank">Android</a> includes all 1000 of our old episodes, as well as related links, greatest hits, and topics. <a href="https://softwareengineeringdaily.com/subscribe" rel="nofollow" target="_blank">Subscribe</a> for ad-free episodes.</li> </ul> The post <a href="https://softwareengineeringdaily.com/2019/11/07/linkedin-data-engineering-with-kapil-surlaker/" rel="nofollow" target="_blank">LinkedIn Data Engineering with Kapil Surlaker</a> appeared first on <a href="https://softwareengineeringdaily.com" rel="nofollow" target="_blank">Software Engineering Daily</a>.