Patrick Cable's guest appearances

Patrick Cable

Data Engineering Podcast

Nikita Shamgunov

Christopher Ryan

Emily Miller

Peter Bull

Patrick Hunt

Jonathan Katz

Kevin Moore

Dan Robinson

Matthew Rocklin

Maxime Beauchemin

Jean Georges Perrin

Todd Walter

Mark Marinelli

Andy LoPresto

Joe Crobak

OwlTail

Pete Cheslock

Summary Cloud computing and ubiquitous virtualization have changed the ways that our applications are built and deployed. This new environment requires a new way of tracking and addressing the security of our systems. ThreatStack is a platform that collects all of the data that your servers generate and monitors for unexpected anomalies in behavior that would indicate a breach and notifies you in near-realtime. In this episode ThreatStack’s director of operations, Pete Cheslock, and senior infrastructure security engineer, Patrick Cable, discuss the data infrastructure that supports their platform, how they capture and process the data from client systems, and how that information can be used to keep your systems safe from attackers. Preamble  Hello and welcome to the Data Engineering Podcast, the show about modern data management When you’re ready to build your next pipeline you’ll need somewhere to deploy it, so check out Linode. With private networking, shared block storage, node balancers, and a 40Gbit network, all controlled by a brand new API you’ve got everything you need to run a bullet-proof data platform. Go to dataengineeringpodcast.com/linode to get a $20 credit and launch a new server in under a minute. For complete visibility into the health of your pipeline, including deployment tracking, and powerful alerting driven by machine-learning, DataDog has got you covered. With their monitoring, metrics, and log collection agent, including extensive integrations and distributed tracing, you’ll have everything you need to find and fix performance bottlenecks in no time. Go to dataengineeringpodcast.com/datadog today to start your free 14 day trial and get a sweet new T-Shirt. Go to dataengineeringpodcast.com to subscribe to the show, sign up for the newsletter, read the show notes, and get in touch. Your host is Tobias Macey and today I’m interviewing Pete Cheslock and Pat Cable about the data infrastructure and security controls at ThreatStack  Interview  Introduction How did you get involved in the area of data management? Why don’t you start by explaining what ThreatStack does?  What was lacking in the existing options (services and self-hosted/open source) that ThreatStack solves for?   Can you describe the type(s) of data that you collect and how it is structured? What is the high level data infrastructure that you use for ingesting, storing, and analyzing your customer data?  How do you ensure a consistent format of the information that you receive? How do you ensure that the various pieces of your platform are deployed using the proper configurations and operating as intended? How much configuration do you provide to the end user in terms of the captured data, such as sampling rate or additional context?   I understand that your original architecture used RabbitMQ as your ingest mechanism, which you then migrated to Kafka. What was your initial motivation for that change?  How much of a benefit has that been in terms of overall complexity and cost (both time and infrastructure)?   How do you ensure the security and provenance of the data that you collect as it traverses your infrastructure? What are some of the most common vulnerabilities that you detect in your client’s infrastructure? For someone who wants to start using ThreatStack, what does the setup process look like? What have you found to be the most challenging aspects of building and managing the data processes in your environment? What are some of the projects that you have planned to improve the capacity or capabilities of your infrastructure?  Contact Info  Pete Cheslock  @petecheslock on Twitter Website petecheslock on GitHub   Patrick Cable  @patcable on Twitter Website patcable on GitHub   ThreatStack  Website @threatstack on Twitter threatstack on GitHub    Parting Question  From your perspective, what is the biggest gap in the tooling or technology for data management today?  Links  ThreatStack SecDevOps Sonian EC2 Snort Snorby Suricata Tripwire Syscall (System Call) AuditD CloudTrail Naxsi Cloud Native File Integrity Monitoring (FIM) Amazon Web Services (AWS) RabbitMQ ZeroMQ Kafka Spark Slack PagerDuty JSON Microservices Cassandra ElasticSearch Sensu Service Discovery Honeypot Kubernetes PostGreSQL Druid Flink Launch Darkly Chef Consul Terraform CloudFormation  The intro and outro music is from The Hug by The Freak Fandango Orchestra  / CC BY-SASupport Data Engineering Podcast

<h2>Summary</h2> <p>Cloud computing and ubiquitous virtualization have changed the ways that our applications are built and deployed. This new environment requires a new way of tracking and addressing the security of our systems. ThreatStack is a platform that collects all of the data that your servers generate and monitors for unexpected anomalies in behavior that would indicate a breach and notifies you in near-realtime. In this episode ThreatStack’s director of operations, Pete Cheslock, and senior infrastructure security engineer, Patrick Cable, discuss the data infrastructure that supports their platform, how they capture and process the data from client systems, and how that information can be used to keep your systems safe from attackers.</p> <h2>Preamble</h2> <ul> <li>Hello and welcome to the Data Engineering Podcast, the show about modern data management</li> <li>When you’re ready to build your next pipeline you’ll need somewhere to deploy it, so check out Linode. With private networking, shared block storage, node balancers, and a 40Gbit network, all controlled by a brand new API you’ve got everything you need to run a bullet-proof data platform. Go to <a href="https://www.dataengineeringpodcast.com/linode?utm_source=rss&amp;utm_medium=rss" rel="nofollow" target="_blank">dataengineeringpodcast.com/linode</a> to get a $20 credit and launch a new server in under a minute.</li> <li>For complete visibility into the health of your pipeline, including deployment tracking, and powerful alerting driven by machine-learning, DataDog has got you covered. With their monitoring, metrics, and log collection agent, including extensive integrations and distributed tracing, you’ll have everything you need to find and fix performance bottlenecks in no time. Go to <a href="https://www.dataengineeringpodcast.com/datadog?utm_source=rss&amp;utm_medium=rss" rel="nofollow" target="_blank">dataengineeringpodcast.com/datadog</a> today to start your free 14 day trial and get a sweet new T-Shirt.</li> <li>Go to <a href="https://www.dataengineeringpodcast.com?utm_source=rss&amp;utm_medium=rss" rel="nofollow" target="_blank">dataengineeringpodcast.com</a> to subscribe to the show, sign up for the newsletter, read the show notes, and get in touch.</li> <li>Your host is Tobias Macey and today I’m interviewing Pete Cheslock and Pat Cable about the data infrastructure and security controls at ThreatStack</li> </ul> <h2>Interview</h2> <ul> <li>Introduction</li> <li>How did you get involved in the area of data management?</li> <li>Why don’t you start by explaining what ThreatStack does? <ul> <li>What was lacking in the existing options (services and self-hosted/open source) that ThreatStack solves for?</li> </ul> <p></p></li> <li>Can you describe the type(s) of data that you collect and how it is structured?</li> <li>What is the high level data infrastructure that you use for ingesting, storing, and analyzing your customer data?<p></p> <ul> <li>How do you ensure a consistent format of the information that you receive?</li> <li>How do you ensure that the various pieces of your platform are deployed using the proper configurations and operating as intended?</li> <li>How much configuration do you provide to the end user in terms of the captured data, such as sampling rate or additional context?</li> </ul> <p></p></li> <li>I understand that your original architecture used RabbitMQ as your ingest mechanism, which you then migrated to Kafka. What was your initial motivation for that change?<p></p> <ul> <li>How much of a benefit has that been in terms of overall complexity and cost (both time and infrastructure)?</li> </ul> <p></p></li> <li>How do you ensure the security and provenance of the data that you collect as it traverses your infrastructure?</li> <li>What are some of the most common vulnerabilities that you detect in your client’s infrastructure?</li> <li>For someone who wants to start using ThreatStack, what does the setup process look like?</li> <li>What have you found to be the most challenging aspects of building and managing the data processes in your environment?</li> <li>What are some of the projects that you have planned to improve the capacity or capabilities of your infrastructure?</li> </ul><p></p> <h2>Contact Info</h2> <ul> <li>Pete Cheslock <ul> <li><a href="https://twitter.com/petecheslock?utm_source=rss&amp;utm_medium=rss" rel="noopener nofollow" target="_blank">@petecheslock</a> on Twitter</li> <li><a href="https://pete.wtf/?utm_source=rss&amp;utm_medium=rss" rel="noopener nofollow" target="_blank">Website</a></li> <li><a href="https://github.com/petecheslock?utm_source=rss&amp;utm_medium=rss" rel="noopener nofollow" target="_blank">petecheslock</a> on GitHub</li> </ul> <p></p></li> <li>Patrick Cable<p></p> <ul> <li><a href="https://twitter.com/patcable?utm_source=rss&amp;utm_medium=rss" rel="noopener nofollow" target="_blank">@patcable</a> on Twitter</li> <li><a href="https://pcable.net/?utm_source=rss&amp;utm_medium=rss" rel="noopener nofollow" target="_blank">Website</a></li> <li><a href="https://github.com/patcable?utm_source=rss&amp;utm_medium=rss" rel="noopener nofollow" target="_blank">patcable</a> on GitHub</li> </ul> <p></p></li> <li>ThreatStack<p></p> <ul> <li><a href="https://www.threatstack.com/?utm_source=rss&amp;utm_medium=rss" rel="noopener nofollow" target="_blank">Website</a></li> <li><a href="https://www.threatstack.com/?utm_source=rss&amp;utm_medium=rss" rel="noopener nofollow" target="_blank">@threatstack</a> on Twitter</li> <li><a href="https://github.com/threatstack?utm_source=rss&amp;utm_medium=rss" rel="noopener nofollow" target="_blank">threatstack</a> on GitHub</li> </ul> <p></p></li> </ul><p></p> <h2>Parting Question</h2> <ul> <li>From your perspective, what is the biggest gap in the tooling or technology for data management today?</li> </ul> <h2>Links</h2> <ul> <li><a href="https://www.threatstack.com/?utm_source=rss&amp;utm_medium=rss" rel="noopener nofollow" target="_blank">ThreatStack</a></li> <li><a href="https://blog.sqreen.io/secdevops/?utm_source=rss&amp;utm_medium=rss" rel="noopener nofollow" target="_blank">SecDevOps</a></li> <li><a href="https://www.sonian.com/?utm_source=rss&amp;utm_medium=rss" rel="noopener nofollow" target="_blank">Sonian</a></li> <li><a href="https://aws.amazon.com/ec2/?utm_source=rss&amp;utm_medium=rss" rel="noopener nofollow" target="_blank">EC2</a></li> <li><a href="https://www.snort.org/?utm_source=rss&amp;utm_medium=rss" rel="noopener nofollow" target="_blank">Snort</a></li> <li><a href="https://github.com/Snorby/snorby?utm_source=rss&amp;utm_medium=rss" rel="noopener nofollow" target="_blank">Snorby</a></li> <li><a href="https://suricata-ids.org/?utm_source=rss&amp;utm_medium=rss" rel="noopener nofollow" target="_blank">Suricata</a></li> <li><a href="https://www.tripwire.com/?utm_source=rss&amp;utm_medium=rss" rel="noopener nofollow" target="_blank">Tripwire</a></li> <li><a href="https://en.wikipedia.org/wiki/System_call?utm_source=rss&amp;utm_medium=rss" rel="noopener nofollow" target="_blank">Syscall (System Call)</a></li> <li><a href="https://www.systutorials.com/docs/linux/man/8-auditd/?utm_source=rss&amp;utm_medium=rss" rel="noopener nofollow" target="_blank">AuditD</a></li> <li><a href="https://aws.amazon.com/cloudtrail/?utm_source=rss&amp;utm_medium=rss" rel="noopener nofollow" target="_blank">CloudTrail</a></li> <li><a href="https://github.com/nbs-system/naxsi?utm_source=rss&amp;utm_medium=rss" rel="noopener nofollow" target="_blank">Naxsi</a></li> <li><a href="https://www.cncf.io/blog/2017/05/15/developing-cloud-native-applications/?utm_source=rss&amp;utm_medium=rss" rel="noopener nofollow" target="_blank">Cloud Native</a></li> <li><a href="https://en.wikipedia.org/wiki/File_integrity_monitoring?utm_source=rss&amp;utm_medium=rss" rel="noopener nofollow" target="_blank">File Integrity Monitoring (FIM)</a></li> <li><a href="https://aws.amazon.com/?utm_source=rss&amp;utm_medium=rss" rel="noopener nofollow" target="_blank">Amazon Web Services (AWS)</a></li> <li><a href="https://www.rabbitmq.com/?utm_source=rss&amp;utm_medium=rss" rel="noopener nofollow" target="_blank">RabbitMQ</a></li> <li><a href="http://zeromq.org/?utm_source=rss&amp;utm_medium=rss" rel="noopener nofollow" target="_blank">ZeroMQ</a></li> <li><a href="http://kafka.apache.org/?utm_source=rss&amp;utm_medium=rss" rel="noopener nofollow" target="_blank">Kafka</a></li> <li><a href="http://spark.apache.org/?utm_source=rss&amp;utm_medium=rss" rel="noopener nofollow" target="_blank">Spark</a></li> <li><a href="https://slack.com/?utm_source=rss&amp;utm_medium=rss" rel="noopener nofollow" target="_blank">Slack</a></li> <li><a href="https://www.pagerduty.com/?utm_source=rss&amp;utm_medium=rss" rel="noopener nofollow" target="_blank">PagerDuty</a></li> <li><a href="http://json.org/?utm_source=rss&amp;utm_medium=rss" rel="noopener nofollow" target="_blank">JSON</a></li> <li><a href="http://microservices.io/?utm_source=rss&amp;utm_medium=rss" rel="noopener nofollow" target="_blank">Microservices</a></li> <li><a href="http://cassandra.apache.org/?utm_source=rss&amp;utm_medium=rss" rel="noopener nofollow" target="_blank">Cassandra</a></li> <li><a href="https://github.com/elastic/elasticsearch?utm_source=rss&amp;utm_medium=rss" rel="noopener nofollow" target="_blank">ElasticSearch</a></li> <li><a href="https://sensuapp.org/?utm_source=rss&amp;utm_medium=rss" rel="noopener nofollow" target="_blank">Sensu</a></li> <li><a href="https://en.wikipedia.org/wiki/Service_discovery?utm_source=rss&amp;utm_medium=rss" rel="noopener nofollow" target="_blank">Service Discovery</a></li> <li><a href="https://en.wikipedia.org/wiki/Honeypot_(computing)?utm_source=rss&amp;utm_medium=rss" rel="noopener nofollow" target="_blank">Honeypot</a></li> <li><a href="https://kubernetes.io/?utm_source=rss&amp;utm_medium=rss" rel="noopener nofollow" target="_blank">Kubernetes</a></li> <li><a href="https://www.postgresql.org/?utm_source=rss&amp;utm_medium=rss" rel="noopener nofollow" target="_blank">PostGreSQL</a></li> <li><a href="http://druid.io/?utm_source=rss&amp;utm_medium=rss" rel="noopener nofollow" target="_blank">Druid</a></li> <li><a href="http://flink.apache.org/?utm_source=rss&amp;utm_medium=rss" rel="noopener nofollow" target="_blank">Flink</a></li> <li><a href="https://launchdarkly.com/?utm_source=rss&amp;utm_medium=rss" rel="noopener nofollow" target="_blank">Launch Darkly</a></li> <li><a href="https://www.chef.io/?utm_source=rss&amp;utm_medium=rss" rel="noopener nofollow" target="_blank">Chef</a></li> <li><a href="https://www.consul.io/?utm_source=rss&amp;utm_medium=rss" rel="noopener nofollow" target="_blank">Consul</a></li> <li><a href="https://www.terraform.io/?utm_source=rss&amp;utm_medium=rss" rel="noopener nofollow" target="_blank">Terraform</a></li> <li><a href="https://aws.amazon.com/cloudformation/?utm_source=rss&amp;utm_medium=rss" rel="noopener nofollow" target="_blank">CloudFormation</a></li> </ul> <p>The intro and outro music is from <a href="http://freemusicarchive.org/music/The_Freak_Fandango_Orchestra/Love_death_and_a_drunken_monkey/04_-_The_Hug?utm_source=rss&amp;utm_medium=rss" rel="noopener nofollow" target="_blank">The Hug</a> by <a href="http://freemusicarchive.org/music/The_Freak_Fandango_Orchestra/?utm_source=rss&amp;utm_medium=rss" rel="noopener nofollow" target="_blank">The Freak Fandango Orchestra</a>  / <a href="http://creativecommons.org/licenses/by-sa/3.0/?utm_source=rss&amp;utm_medium=rss" rel="noopener nofollow" target="_blank">CC BY-SA</a></p><p><a href="https://dataengineering.supercast.com/" rel="payment nofollow" target="_blank">Support Data Engineering Podcast</a></p>

ThreatStack: Data Driven Cloud Security with Pete Cheslock and Patrick Cable - Episode 25