Join us at the DoK Town Hall for an insightful session on empowering developers with easy, scalable stream processing on Kubernetes (K8s). Discover the latest tools and techniques for building efficient, real-time data pipelines. Our expert speakers will share best practices, live demos, and success stories, helping you harness the full potential of K8s for your stream processing needs. Don't miss this opportunity to enhance your skills and network with industry peers!

We are the Data on Kubernetes Community (DoKC), where end users go to share best practices for running data workloads on Kubernetes.

We're excited to host our monthly DoKC Town Hall virtual event in September 2024! This is an event to bring the community together to meet each other, share end-user journey stories, DoK-related projects and technologies, and keep you up-to-date on community events and ways to participate. Meetings will be held on the third Thursday of each month at 10am PT.

RSVP: https://www.meetup.com/data-on-kubernetes-community/events/302440395/

AGENDA

[10:00 AM]

Welcome and Community Updates

Presented by Paul Au, Head of Community

[10:05 AM]

Empowering developers with easy, scalable stream processing on Kubernetes

Speakers:

Sri Yayi, Senior Product Manager, Intuit

Vigith Maurice Principle Engineer, Intuit

Description:

Stream processing and data analytics are needed by both data engineers and non-data engineers such as platform engineers, DevOps, etc. How to make real-time stream data processing easy to use, cost-efficient and resilient to pod restarts or node upgrades is a big challenge. While there are existing stream processing solutions, they require a steep learning curve, are operationally intensive and usually costly. This talk will share our experience building a generic open-source K8s native stream processing framework called Numaflow. It enables developers to easily and quickly run large-scale stream processing jobs without needing to depend on heavy and costly data processing platforms; Using this platform, Intuit's application developers process ~5B messages for analytics, ML engineers train 135K models and make 60M predictions daily. It's been powering Intuit's large scale anomaly detection platform running in 200+ Kubernetes clusters

[10:55 AM]

DoK Quiz