You must manage GenAI traffic to leverage GenAI services, especially from multiple providers, which presents specific challenges. Although GenAI traffic may initially seem similar to regular API traffic, how we integrate with providers, monitor, route, and the fundamental nature of AI traffic differ. Envoy AI Gateway was initiated as an open-source solution as part of the Envoy project to address the gaps we saw from these differences, bringing the community together to enable and simplify the handling of GenAI traffic.

In this CNCF Live session, you will learn about the Envoy AI Gateway, the community surrounding it, their ongoing work, and how you can get involved. There will be some amusing trivia as we show how Envoy AI Gateway works and what it does, and we will share what to expect in the upcoming 0.2 release.

We look forward to continuing this journey with you all. Built together. Built in the open.