Core Developer Fabian Reinartz on the Prometheus 2.0 storage engine

Cloudflare - 25 Lavington St, SE1 0NZ London - View Map Prometheus London User Group
Wed, Jun 14, 2017, 6:30 PM (BST)

About this event

After a short hiatus, we're pleased to announce a June Prometheus London meetup.


Fabian Reinartz, core Prometheus developer and software engineer at CoreOS, will be talking about the new storage engine in Prometheus 2.0.

Tom Wilkie, software engineer at Weaveworks, will be giving a talk (originally scheduled for our last meetup, but postponed) on Weaveworks Cortex, an open source and horizontally-scalable project based on Prometheus.

We'll be finishing with 5-minute lightning talks. Please volunteer!

If you'd like to give a lightning talk, please send a message to Matt with a brief outline of your talk. Lightning talks should be 5 minutes long. If you haven't given a talk before, this is a great opportunity to do so.

Fabian Reinartz: The new Prometheus storage engine

From the beginning, Prometheus was built as a monitoring system with Cloud Native environments in mind. 

Orchestration systems such as Kubernetes are rapidly gaining traction and unlock features of highly dynamic environments, such as frequent rolling updates and auto-scaling, for everyone. This inevitably puts new strains on Prometheus as well.

In this talk we explore what the challenges are and how we are addressing them by building a new storage layer from the ground up.

The new design enables consistent performance with thousands of concurrent writers inserting millions of new samples per second on a single machine.

We will discuss how the new approach significantly reduces resource requirements for CPU, memory, and disk IO alike. At the same time its more efficient indexing techniques increase query performance and allow us to gracefully handle high turnover rates of monitored application instances.

Bio: Fabian is a software engineer at CoreOS, where he works on the Prometheus open source eco-system and its integration with Kubernetes. Previously, he was an engineer at SoundCloud.

Tom Wilkie: Cortex: open-source, horizontally-scalable, distributed Prometheus

In this talk we'll present a solution for a horizontally
scalable, distributed Prometheus implementation dubbed "Cortex" (née Frankenstein). 

Cortex turns a lot of the Prometheus architectural assumptions on its head, by marrying a scale-out PromQL query engine with a cloud-native storage layer based on DynamoDB. Cortex not only add horizontal scalability in terms of ingest rate and active timeseries, but also adds virtually infinite data retention and some interesting opportunities for query optimisation.

Bio: Tom is a Software Engineer at Weaveworks. Previously he was at Google as a Site Reliability Manager for Google Analytics. Before that he was Founder, VP Eng and CTO at Acunu, and before that a Software Engineer at XenSource. In his spare time, Tom likes to make craft beer and build home automation systems.


Wednesday, Jun 14
6:30 PM - 9:30 PM (BST)


25 Lavington St, SE1 0NZ London