Kapil Surlaker's guest appearances

Kapil Surlaker

Software Engineering Daily

Data Archives - Software Engineering Daily

Podcast Archives - Software Engineering Daily

Software Daily

Joe Doliner

Surabhi Gupta

Eran Zinman

Matthew Seal

Soren Bramer Schmidt

Paul Banks

David Potere

Joe Hellerstein

George Fraser

Howie Liu

Raj Bains

Abe Gong

Dhruba Borthakur

Steve Newman

Kevin Novak

OwlTail

A large social network needs to develop systems for ingesting, storing, and processing large volumes of data.Data engineering at scale requires multiple engineering teams that are responsible for different areas of the infrastructure.Data needs to be structured coherently in order to minimize the data cleaning process. Machine learning models need to be developed, deployed, and iterated on at scale. Areas of the company which produce data need to be decoupled from the areas of the company which consume data, so that engineers throughout the company can reliably build tools on top of these large data sets.In our previous episodes about LinkedIn, we covered two major components of LinkedIn’s data engineering systems: the Kafka infrastructure and the LinkedIn data platform used by engineers to productively build data applications.Kapil Surlaker is a senior director of engineering at LinkedIn, and he joins the show to discuss the bigger picture of LinkedIn’s data infrastructure. Kapil works with teams across LinkedIn to understand the requirements for the products and internal tools, and translate those requirements into team structures and software platforms that let LinkedIn use data more productively.We discuss a wide range of topics, including engineering management, the modern data platform, and LinkedIn’s adoption of public cloud.Full disclosure: LinkedIn is a sponsor of Software Engineering Daily. 

A large social network needs to develop systems for ingesting, storing, and processing large volumes of data.Data engineering at scale requires multiple engineering teams that are responsible for different areas of the infrastructure.Data needs to be structured coherently in order to minimize the data cleaning process. Machine learning models need to be developed, deployed, and iterated on at scale. Areas of the company which produce data need to be decoupled from the areas of the company which consume data, so that engineers throughout the company can reliably build tools on top of these large data sets.In our previous episodes about LinkedIn, we covered two major components of LinkedIn’s data engineering systems: the Kafka infrastructure and the LinkedIn data platform used by engineers to productively build data applications.Kapil Surlaker is a senior director of engineering at LinkedIn, and he joins the show to discuss the bigger picture of LinkedIn’s data infrastructure. Kapil works with teams across LinkedIn to understand the requirements for the products and internal tools, and translate those requirements into team structures and software platforms that let LinkedIn use data more productively.We discuss a wide range of topics, including engineering management, the modern data platform, and LinkedIn’s adoption of public cloud.Full disclosure: LinkedIn is a sponsor of Software Engineering Daily.

LinkedIn Data Engineering with Kapil Surlaker

A large social network needs to develop systems for ingesting, storing, and processing large volumes of data. Data engineering at scale requires multiple engineering teams that are responsible for different areas of the infrastructure. Data needs to be structured coherently in order to minimize the data cleaning process. Machine learning models need to be developed, deployed, and iterated on at scale. Areas of the company which produce data need to be decoupled from the areas of the company which consume data, so that engineers throughout the company can reliably build tools on top of these large data sets. In our previous episodes about LinkedIn, we covered two major components of LinkedIn’s data engineering systems: the Kafka infrastructure and the LinkedIn data platform used by engineers to productively build data applications. Kapil Surlaker is a senior director of engineering at LinkedIn, and he joins the show to discuss the bigger picture of LinkedIn’s data infrastructure. Kapil works with teams across LinkedIn to understand the requirements for the products and internal tools, and translate those requirements into team structures and software platforms that let LinkedIn use data more productively. We discuss a wide range of topics, including engineering management, the modern data platform, and LinkedIn’s adoption of public cloud. Full disclosure: LinkedIn is a sponsor of Software Engineering Daily.  Sponsorship inquiries: sponsor@softwareengineeringdaily.com Check out our active projects:  We are hiring a head of growth. If you like Software Engineering Daily and consider yourself competent in sales, marketing, and strategy, send me an email: jeff@softwareengineeringdaily.com FindCollabs is a place to build open source software. The SEDaily app for iOS and Android includes all 1000 of our old episodes, as well as related links, greatest hits, and topics. Subscribe for ad-free episodes.  The post LinkedIn Data Engineering with Kapil Surlaker appeared first on Software Engineering Daily.

A large social network needs to develop systems for ingesting, storing, and processing large volumes of data. Data engineering at scale requires multiple engineering teams that are responsible for different areas of the infrastructure. Data needs to be structured coherently in order to minimize the data cleaning process. Machine learning models need to be developed, deployed, and iterated on at scale. Areas of the company which produce data need to be decoupled from the areas of the company which consume data, so that engineers throughout the company can reliably build tools on top of these large data sets. In our previous episodes about LinkedIn, we covered two major components of LinkedIn’s data engineering systems: the Kafka infrastructure and the LinkedIn data platform used by engineers to productively build data applications. Kapil Surlaker is a senior director of engineering at LinkedIn, and he joins the show to discuss the bigger picture of LinkedIn’s data infrastructure. Kapil works with teams across LinkedIn to understand the requirements for the products and internal tools, and translate those requirements into team structures and software platforms that let LinkedIn use data more productively. We discuss a wide range of topics, including engineering management, the modern data platform, and LinkedIn’s adoption of public cloud. Full disclosure: LinkedIn is a sponsor of Software Engineering Daily.  Sponsorship inquiries: <a href="mailto:sponsor@softwareengineeringdaily.com">sponsor@softwareengineeringdaily.com</a> <h2>Check out our active projects:</h2> <ul> <li>We are hiring a <a href="https://www.softwaredaily.com/jobs/5c3a6cf0f7fb250029d873a9" rel="nofollow" target="_blank">head of growth</a>. If you like Software Engineering Daily and consider yourself competent in sales, marketing, and strategy, send me an email: jeff@softwareengineeringdaily.com</li> <li><a href="https://findcollabs.com/" rel="nofollow" target="_blank">FindCollabs</a> is a place to build open source software.</li> <li><a href="https://itunes.apple.com/us/app/software-engineering-daily/id1253734426?mt=8" rel="nofollow" target="_blank">The SEDaily app for iOS</a> and <a href="https://play.google.com/store/apps/details?id=com.koalatea.sedaily" rel="nofollow" target="_blank">Android</a> includes all 1000 of our old episodes, as well as related links, greatest hits, and topics. <a href="https://softwareengineeringdaily.com/subscribe" rel="nofollow" target="_blank">Subscribe</a> for ad-free episodes.</li> </ul> The post <a href="https://softwareengineeringdaily.com/2019/11/07/linkedin-data-engineering-with-kapil-surlaker/" rel="nofollow" target="_blank">LinkedIn Data Engineering with Kapil Surlaker</a> appeared first on <a href="https://softwareengineeringdaily.com" rel="nofollow" target="_blank">Software Engineering Daily</a>.