DoK #51 Promscale: Using Prometheus + Promscale + PostgreSQL to go from Observation to Understanding

Data on Kubernetes
Thu, May 27, 9:00 AM (PDT)

About this event

Abstract of the talkā€¦

Often when I talk about putting observability data into PostgreSQL people ask me: are you crazy? And yet this somewhat heretical view has the potential to unlock a lot of the power and promise of observability. Thanks to TimescaleDB (an extension to PostgreSQL), storing time series, metric data inside of a relational database is now efficient, fast, and scalable. This is thanks to its unique partitioning, compression, and horizontal-scalability features. But even if this is possible to do, why would you? The answer to that lies in the power of a flexible data model, joins, and SQL (which Promscale supports in addition to PromQL). A flexible data model allows you to combine metric data with various other data - from machine information such as the number of cores, memory, etc. to location information using GPS coordinates. This allows you to enrich your metrics with supplemental information using joins and performing much more sophisticated analysis using SQL for capacity analysis, BI, and more. A flexible data model brings us to our second heretical idea: combining multiple modalities of observation in a single database. Combining metrics, logs, traces, event data, etc. in one DB has two major advantages: the first being a similar analytical advantage to what is described above: the ability to join and cross-correlate various types of signals together. The second major advantage is operational simplicity. As we all know, databases are the hardest things in our infrastructure to maintain and operationalize because of that pesky thing called state. So why maintain multiple different types of database systems if you could maintain just one? While these ideas about observability data on Kubernetes may seem unusual and counter-intuitive, I hope they will generate interest and start a good conversation.

Speaker

  • Matvey Arye

    Matvey Arye

    Promscale

    Promscale Team Lead

    Mat has been working on data infrastructure in both academia and industry for the past decade. Currently, he is leading the Promscale team, to make it easy for people to store and analyze their Prometheus data in both PromQL and SQL. Previously, he completed his Ph.D. at Princeton and then worked as one of TimescaleDB's core architects where he concentrated on performance, scalability, and quer...

    Read More

  • Organizers

  • Ihor Dvoretskyi

    Ihor Dvoretskyi

    Cloud Native Computing Foundation

    Organizer

    View Profile
  • Bart Farrell

    Bart Farrell

    Data on Kubernetes Community

    Organizer

    View Profile
  • Melissa Logan

    Melissa Logan

    Organizer

    View Profile
  • Diogenese Topper

    Diogenese Topper

    Organizer

    View Profile
  • Iker Arce

    Iker Arce

    Organizer

    View Profile