An Update on PromQL Compatibility Across Vendors

An Update on PromQL Compatibility Across Vendors(promlabs.com)

68 points by jrv 5 years ago | 12 comments

Out of all the projects/products on this list, we've found Victoria Metrics to be by _far_ the best.

This summary makes it seem like Victoria Metrics is barely compatible with Prometheus, but that can't be further from the truth in practice.

PeterZaitsev 5 years ago | |

This is what is the problem with such compatibility tests... they tend to test full power of the language while you really may use quite small subset in your application, as such even solution which is "20% compatible" may well meet all your application needs.

I remember in its early days MySQL had pretty poor SQL support (if you think about full standard) which did not prevent it from having huge success.

Or more recent example ClickHouse which I think similar to VictoriaMetrics as it does not fully implement SQL, but also adds many convenient extensions which are not part of the standard.

Chances are if you chose VictoriaMetrics you will find a lot more utility in advanced features of MetricsQL than you loose from exact compatibility with PromQL

EdwardDiego 5 years ago | |

Did you start with Victoria directly? Or move to it from Prometheus? Keen to hear your experiences.

We're currently running Prometheus + Thanos, and high cardinality timeseries are a real issue, which Victoria claims to be good at.

shitloadofbooks 5 years ago | | |

We went from Prometheus (bursting at the seams and begging to be scaled) -> a failed Cortex implementation -> a rock solid Victoria Metrics implementation.

We found Cortex to be extremely over-engineered, extremely hard to tune (because of multiple configuration/argument refactors with incomplete doc cleanups) and found that it would just fall over under load. (Though to be fair, there wasn't much in the way of first-party deployment tooling at the time, so it was a hand-rolled Helm chart and at least 50% of the issues were likely my config).

In comparison, the first-party Victoria Metrics Helm chart worked straight out of the box, the maintainers have fixed multiple small issues within hours of me reporting them and we've thrown an extremely large amount of metrics at it with zero problems.

cevian 5 years ago | | |

(Promscale Team Lead Here) Promscale, which connects Prometheus to TimescaleDB also handles high-cardinality well and allows you to use both PromQL and SQL for data analysis. I'd humbly suggest taking a look. Plus our PromQL implementation is 100% standards compliant.

pointestimate 5 years ago |

Prometheus can't even do bulk import.

How do you include that in a CI/CD environment with regression testing?

benraskin92 5 years ago |

This is great -- thanks for putting it together! Will these be kept up-to-date? If so, what will the cadence be?

jrv 5 years ago | |

Hi! On the road right now, but back later :) There's no established cadence yet, but twice a year or so sounds like a reasonable update period to me.

valyala 5 years ago |

VictoriaMetrics CTO here.

These tests are great, because they spotted a few minor bugs in our PromQL implementation and these bugs were fixed quickly. See https://victoriametrics.github.io/CHANGELOG.html .

The majority of failed tests for VictoriaMetrics cannot be "fixed" due to deliberate choice made when designing MetricsQL: to rethink and to fix the most annoying and confusing parts of PromQL, while providing drop-in PromQL replacement for the majority of practical cases. See more details at https://victoriametrics.github.io/MetricsQL.html .