Chris Groskopf's guest appearances

Chris Groskopf

Data Engineering Podcast

Business and Philosophy

Software Daily

Rob Walch

Frank Chen

Erik Torenberg

Seth Godin

Ben Horowitz

Bob Hoffman

Vivek Saraswat

Ilya Sukhar

Leo Polovets

Geoff Ralston

Elad Gil

George Anders

Gaurav Jain

Semil Shah

Scott Belsky

OwlTail

Summary There are countless sources of data that are publicly available for use. Unfortunately, combining those sources and making them useful in aggregate is a time consuming and challenging process. The team at Enigma builds a knowledge graph for use in your own data projects. In this episode Chris Groskopf explains the platform they have built to consume large varieties and volumes of public data for constructing a graph for serving to their customers. He discusses the challenges they are facing to scale the platform and engineering processes, as well as the workflow that they have established to enable testing of their ETL jobs. This is a great episode to listen to for ideas on how to organize a data engineering organization. Preamble  Hello and welcome to the Data Engineering Podcast, the show about modern data management When you’re ready to build your next pipeline you’ll need somewhere to deploy it, so check out Linode. With private networking, shared block storage, node balancers, and a 40Gbit network, all controlled by a brand new API you’ve got everything you need to run a bullet-proof data platform. Go to dataengineeringpodcast.com/linode to get a $20 credit and launch a new server in under a minute. You work hard to make sure that your data is reliable and accurate, but can you say the same about the deployment of your machine learning models? The Skafos platform from Metis Machine was built to give your data scientists the end-to-end support that they need throughout the machine learning lifecycle. Skafos maximizes interoperability with your existing tools and platforms, and offers real-time insights and the ability to be up and running with cloud-based production scale infrastructure instantaneously. Request a demo at dataengineeringpodcast.com/metis-machine to learn more about how Metis Machine is operationalizing data science. Go to dataengineeringpodcast.com to subscribe to the show, sign up for the mailing list, read the show notes, and get in touch. Join the community in the new Zulip chat workspace at dataengineeringpodcast.com/chat Your host is Tobias Macey and today I’m interviewing Chris Groskopf about Enigma and how the are using public data sources to build a knowledge graph  Interview  Introduction How did you get involved in the area of data management? Can you give a brief overview of what Enigma has built and what the motivation was for starting the company?  How do you define the concept of a knowledge graph?   What are the processes involved in constructing a knowledge graph? Can you describe the overall architecture of your data platform and the systems that you use for storing and serving your knowledge graph? What are the most challenging or unexpected aspects of building the knowledge graph that you have encountered?  How do you manage the software lifecycle for your ETL code? What kinds of unit, integration, or acceptance tests do you run to ensure that you don’t introduce regressions in your processing logic?   What are the current challenges that you are facing in building and scaling your data infrastructure?  How does the fact that your data sources are primarily public influence your pipeline design and what challenges does it pose? What techniques are you using to manage accuracy and consistency in the data that you ingest?   Can you walk through the lifecycle of the data that you process from acquisition through to delivery to your customers? What are the weak spots in your platform that you are planning to address in upcoming projects?  If you were to start from scratch today, what would you have done differently?   What are  some of the most interesting or unexpected uses of your product that you have seen? What is in store for the future of Enigma?  Contact Info  Email Twitter  Parting Question  From your perspective, what is the biggest gap in the tooling or technology for data management today?  Links  Enigma Chicago Tribune NPR Quartz CSVKit Agate Knowledge Graph Taxonomy Concourse Airflow Docker S3 Data Lake Parquet  Podcast Episode   Spark AWS Neptune AWS Batch Money Laundering Jupyter Notebook Papermill Jupytext Cauldron: The Un-Notebook  Podcast.__init__ Episode    The intro and outro music is from The Hug by The Freak Fandango Orchestra  / CC BY-SA Support Data Engineering Podcast

<h3>Summary</h3> <p>There are countless sources of data that are publicly available for use. Unfortunately, combining those sources and making them useful in aggregate is a time consuming and challenging process. The team at Enigma builds a knowledge graph for use in your own data projects. In this episode Chris Groskopf explains the platform they have built to consume large varieties and volumes of public data for constructing a graph for serving to their customers. He discusses the challenges they are facing to scale the platform and engineering processes, as well as the workflow that they have established to enable testing of their ETL jobs. This is a great episode to listen to for ideas on how to organize a data engineering organization.</p> <h2>Preamble</h2> <ul> <li>Hello and welcome to the Data Engineering Podcast, the show about modern data management</li> <li>When you’re ready to build your next pipeline you’ll need somewhere to deploy it, so check out Linode. With private networking, shared block storage, node balancers, and a 40Gbit network, all controlled by a brand new API you’ve got everything you need to run a bullet-proof data platform. Go to <a href="https://www.dataengineeringpodcast.com/linode?utm_source=rss&amp;utm_medium=rss" rel="nofollow" target="_blank">dataengineeringpodcast.com/linode</a> to get a $20 credit and launch a new server in under a minute.</li> <li>You work hard to make sure that your data is reliable and accurate, but can you say the same about the deployment of your machine learning models? The Skafos platform from Metis Machine was built to give your data scientists the end-to-end support that they need throughout the machine learning lifecycle. Skafos maximizes interoperability with your existing tools and platforms, and offers real-time insights and the ability to be up and running with cloud-based production scale infrastructure instantaneously. Request a demo at <a href="https://www.dataengineeringpodcast.com/metis-machine?utm_source=rss&amp;utm_medium=rss" rel="nofollow" target="_blank">dataengineeringpodcast.com/metis-machine</a> to learn more about how Metis Machine is operationalizing data science.</li> <li>Go to <a href="https://www.dataengineeringpodcast.com?utm_source=rss&amp;utm_medium=rss" rel="nofollow" target="_blank">dataengineeringpodcast.com</a> to subscribe to the show, sign up for the mailing list, read the show notes, and get in touch.</li> <li>Join the community in the new Zulip chat workspace at <a href="https://www.dataengineeringpodcast.com/chat?utm_source=rss&amp;utm_medium=rss" rel="nofollow" target="_blank">dataengineeringpodcast.com/chat</a></li> <li>Your host is Tobias Macey and today I’m interviewing Chris Groskopf about Enigma and how the are using public data sources to build a knowledge graph</li> </ul> <h2>Interview</h2> <ul> <li>Introduction</li> <li>How did you get involved in the area of data management?</li> <li>Can you give a brief overview of what Enigma has built and what the motivation was for starting the company? <ul> <li>How do you define the concept of a knowledge graph?</li> </ul> <p></p></li> <li>What are the processes involved in constructing a knowledge graph?</li> <li>Can you describe the overall architecture of your data platform and the systems that you use for storing and serving your knowledge graph?</li> <li>What are the most challenging or unexpected aspects of building the knowledge graph that you have encountered?<p></p> <ul> <li>How do you manage the software lifecycle for your ETL code?</li> <li>What kinds of unit, integration, or acceptance tests do you run to ensure that you don’t introduce regressions in your processing logic?</li> </ul> <p></p></li> <li>What are the current challenges that you are facing in building and scaling your data infrastructure?<p></p> <ul> <li>How does the fact that your data sources are primarily public influence your pipeline design and what challenges does it pose?</li> <li>What techniques are you using to manage accuracy and consistency in the data that you ingest?</li> </ul> <p></p></li> <li>Can you walk through the lifecycle of the data that you process from acquisition through to delivery to your customers?</li> <li>What are the weak spots in your platform that you are planning to address in upcoming projects?<p></p> <ul> <li>If you were to start from scratch today, what would you have done differently?</li> </ul> <p></p></li> <li>What are  some of the most interesting or unexpected uses of your product that you have seen?</li> <li>What is in store for the future of Enigma?</li> </ul><p></p> <h2>Contact Info</h2> <ul> <li><a href="chris.groskopf@enigma.com" rel="nofollow" target="_blank">Email</a></li> <li><a href="https://twitter.com/onyxfish?utm_source=rss&amp;utm_medium=rss" rel="noopener nofollow" target="_blank">Twitter</a></li> </ul> <h2>Parting Question</h2> <ul> <li>From your perspective, what is the biggest gap in the tooling or technology for data management today?</li> </ul> <h2>Links</h2> <ul> <li><a href="https://www.enigma.com/?utm_source=rss&amp;utm_medium=rss" rel="noopener nofollow" target="_blank">Enigma</a></li> <li><a href="http://www.chicagotribune.com/?utm_source=rss&amp;utm_medium=rss" rel="noopener nofollow" target="_blank">Chicago Tribune</a></li> <li><a href="https://www.npr.org/?utm_source=rss&amp;utm_medium=rss" rel="noopener nofollow" target="_blank">NPR</a></li> <li><a href="https://qz.com/?utm_source=rss&amp;utm_medium=rss" rel="noopener nofollow" target="_blank">Quartz</a></li> <li><a href="https://csvkit.readthedocs.io/en/latest/?utm_source=rss&amp;utm_medium=rss" rel="noopener nofollow" target="_blank">CSVKit</a></li> <li><a href="https://agate.readthedocs.io/?utm_source=rss&amp;utm_medium=rss" rel="noopener nofollow" target="_blank">Agate</a></li> <li><a href="https://en.wikipedia.org/wiki/Knowledge_graph?utm_source=rss&amp;utm_medium=rss" rel="noopener nofollow" target="_blank">Knowledge Graph</a></li> <li><a href="https://en.wikipedia.org/wiki/Taxonomy_(general)?utm_source=rss&amp;utm_medium=rss" rel="noopener nofollow" target="_blank">Taxonomy</a></li> <li><a href="https://www.enigma.com/data-infrastructure/concourse?utm_source=rss&amp;utm_medium=rss" rel="noopener nofollow" target="_blank">Concourse</a></li> <li><a href="http://airflow.apache.org/?utm_source=rss&amp;utm_medium=rss" rel="noopener nofollow" target="_blank">Airflow</a></li> <li><a href="https://www.docker.com/?utm_source=rss&amp;utm_medium=rss" rel="noopener nofollow" target="_blank">Docker</a></li> <li><a href="https://aws.amazon.com/s3/?utm_source=rss&amp;utm_medium=rss" rel="noopener nofollow" target="_blank">S3</a></li> <li><a href="https://en.wikipedia.org/wiki/Data_lake?utm_source=rss&amp;utm_medium=rss" rel="noopener nofollow" target="_blank">Data Lake</a></li> <li><a href="http://parquet.apache.org/?utm_source=rss&amp;utm_medium=rss" rel="noopener nofollow" target="_blank">Parquet</a> <ul> <li><a href="https://www.dataengineeringpodcast.com/data-serialization-with-doug-cutting-and-julien-le-dem-episode-8/?utm_source=rss&amp;utm_medium=rss" rel="nofollow" target="_blank">Podcast Episode</a></li> </ul> <p></p></li> <li><a href="http://spark.apache.org/?utm_source=rss&amp;utm_medium=rss" rel="noopener nofollow" target="_blank">Spark</a></li> <li><a href="https://aws.amazon.com/neptune/?utm_source=rss&amp;utm_medium=rss" rel="noopener nofollow" target="_blank">AWS Neptune</a></li> <li><a href="https://aws.amazon.com/batch/?utm_source=rss&amp;utm_medium=rss" rel="noopener nofollow" target="_blank">AWS Batch</a></li> <li><a href="https://en.wikipedia.org/wiki/Money_laundering?utm_source=rss&amp;utm_medium=rss" rel="noopener nofollow" target="_blank">Money Laundering</a></li> <li><a href="http://jupyter.org/?utm_source=rss&amp;utm_medium=rss" rel="noopener nofollow" target="_blank">Jupyter Notebook</a></li> <li><a href="https://papermill.readthedocs.io/en/latest/?utm_source=rss&amp;utm_medium=rss" rel="noopener nofollow" target="_blank">Papermill</a></li> <li><a href="https://github.com/mwouts/jupytext?utm_source=rss&amp;utm_medium=rss" rel="noopener nofollow" target="_blank">Jupytext</a></li> <li><a href="http://www.unnotebook.com/?utm_source=rss&amp;utm_medium=rss" rel="noopener nofollow" target="_blank">Cauldron: The Un-Notebook</a><p></p> <ul> <li><a href="https://www.podcastinit.com/episode-111-cauldron-notebook-with-scott-ernst/?utm_source=rss&amp;utm_medium=rss" rel="noopener nofollow" target="_blank">Podcast.__init__ Episode</a></li> </ul> <p></p></li> </ul><p></p> <p>The intro and outro music is from <a href="http://freemusicarchive.org/music/The_Freak_Fandango_Orchestra/Love_death_and_a_drunken_monkey/04_-_The_Hug?utm_source=rss&amp;utm_medium=rss" rel="noopener nofollow" target="_blank">The Hug</a> by <a href="http://freemusicarchive.org/music/The_Freak_Fandango_Orchestra/?utm_source=rss&amp;utm_medium=rss" rel="noopener nofollow" target="_blank">The Freak Fandango Orchestra</a>  / <a href="http://creativecommons.org/licenses/by-sa/3.0/?utm_source=rss&amp;utm_medium=rss" rel="noopener nofollow" target="_blank">CC BY-SA</a></p> <p><a href="https://dataengineering.supercast.com/" rel="payment nofollow" target="_blank">Support Data Engineering Podcast</a></p>

Building A Knowledge Graph From Public Data At Enigma With Chris Groskopf - Episode 50

When you purchase an ebook you must agree to the Terms of Service that tell you what you can do with it. What is actually in that terms of service? What are you agreeing to when you buy an ebook? The answers might surprise you. In this episode, Srini Kadamati interviews Chris Groskopf  on how the rise of digital products has eroded the idea of traditional ownership. They discuss digital ownership from the point of view of the legal system, consumers, and the companies creating these products. Chris Groskopf is a data journalist who uses data, graphics, and storytelling to build compelling news experiences. He’s worked on multiple pioneering teams at organizations like the Chicago Tribune and NPR. He’s currently the first data editor at Quartz, a digital-first Atlantic Media publication, where he’s written about how complex systems like the stock market fail and how most of the world’s art is locked away in museums. Outside of journalism, he’s created multiple Python data libraries, like agate, proof, and csvkit. Show Notes Chris Groskopf on Twitter Quartz author page Terms of Service story Perzanowski’s book For Programmers, the Ultimate Office Perk is Avoiding the Office Entirely The post Digital Ownership with Chris Groskopf appeared first on Software Engineering Daily.

<p>When you purchase an ebook you must agree to the Terms of Service that tell you what you can do with it. What is actually in that terms of service? What are you agreeing to when you buy an ebook? The answers might surprise you.</p> <p>In this episode, Srini Kadamati interviews Chris Groskopf  on how the rise of digital products has eroded the idea of traditional ownership. They discuss digital ownership from the point of view of the legal system, consumers, and the companies creating these products.</p> <p>Chris Groskopf is a data journalist who uses data, graphics, and storytelling to build compelling news experiences. He’s worked on multiple pioneering teams at organizations like the <em>Chicago Tribune</em> and NPR. He’s currently the first data editor at <em>Quartz</em>, a digital-first Atlantic Media publication, where he’s written about how complex systems like the stock market fail and how most of the world’s art is locked away in museums. Outside of journalism, he’s created multiple Python data libraries, like agate, proof, and csvkit.</p> <h2>Show Notes</h2> <a href="https://twitter.com/onyxfish" rel="noopener noreferrer nofollow" target="_blank">Chris Groskopf on Twitter</a> <a href="https://www.theatlas.com/user/cgroskopf" rel="noopener noreferrer nofollow" target="_blank">Quartz author page</a> <a href="https://qz.com/766535/terms-of-service-agreements-are-destroying-the-concept-of-ownership-for-digital-goods/" rel="noopener noreferrer nofollow" target="_blank">Terms of Service story</a> <a href="https://www.amazon.com/End-Ownership-Personal-Property-Information-ebook/dp/B01MFFNYC4/ref=sr_1_sc_1?ie=UTF8&amp;qid=1493817294&amp;sr=8-1-spell&amp;keywords=perznowski" rel="noopener noreferrer nofollow" target="_blank">Perzanowski’s book</a> <a href="https://qz.com/950973/remote-work-for-programmers-the-ultimate-office-perk-is-avoiding-the-office-entirely/" rel="nofollow" target="_blank">For Programmers, the Ultimate Office Perk is Avoiding the Office Entirely</a> <p>The post <a href="https://softwareengineeringdaily.com/2017/05/16/digital-ownership-with-chris-groskopf/" rel="nofollow" target="_blank">Digital Ownership with Chris Groskopf</a> appeared first on <a href="https://softwareengineeringdaily.com" rel="nofollow" target="_blank">Software Engineering Daily</a>.</p>

Digital Ownership with Chris Groskopf