PromQL queries inGoogle Cloud Managed Service for Prometheus are partially evaluated at theMonarch backend, and there are some known differencesin query results. This document describes the differences.

Other than the differences listed in thisdocument,the PromQL in Managed Service for Prometheus is at paritywith the PromQL available in Prometheus version2.44.

PromQL functions added after Prometheus version2.44 might not besupported.

UTF-8 support

PromQL for Cloud Monitoring supportsUTF-8 querying.

If your Prometheus metric name only consists of alphanumeric characters plus the_ or:characters, and if your label keys only consist of alphanumeric characters plusthe_ character, then you can query using traditional PromQL syntax.For example, a valid query might look likejob:my_metric:sum{label_key="label_value"}.

However, if your Prometheus metric name uses any special charactersexcept for the_ or: characters, or if your label keys use any specialcharacter except for the_ character, then you have to construct your queryaccording to theUTF-8 spec for PromQL.

UTF-8 metric names must be quoted and moved into the braces. Label names mustalso be quoted if they contain legacy-incompatible characters. The followingexample valid queries are all equivalent:

• {"my.domain.com/metric/name_bucket", "label.key"="label.value"}
• {__name__="my.domain.com/metric/name_bucket", "label.key"="label.value"}
• {"__name__"="my.domain.com/metric/name_bucket", "label.key"="label.value"}

Matching on metric names

Only exact matching on metric names is supported. You must includean exact match on the metric name in your query.

We recommend the following workarounds for common scenarios that use a regularexpression matcher on the__name__ label:

• Prometheus adapter configurations often use the=~ operator to match onmultiple metric names. To fix this usage, expand the config to use a separatepolicy for each metric and name each metric explicitly. This alsoprevents you from accidentally autoscaling on unexpected metrics.
• Regular expressions are often used to graph multiple non-dimensional metricson the same chart. For example, if you have a metric likecpu_servicename_usage, you might use a wildcard to graph allyour services together. Using non-dimensional metrics like this is anexplicitly bad practice in Cloud Monitoring, and this practice leads toextremely poor query performance. To fix this usage, move all dimensionalityinto metric labels instead of embedding dimensions in the metric name.
• Querying over multiple metrics is often used to see what metrics areavailable to query. We recommend you instead use the/labels/__name__/values call to discover metrics.You can also discover metrics using the Cloud Monitoring UI.
• Matching multiple metrics is useful for seeing how many sampleswere scraped, ingested, and charged on a per-metric basis. Cloud Monitoringprovides this information to you on theMetrics Managementpage. You can also access this information as metric data by using theSamples Ingested metric or the Samples Written by Attribution IDmetric.

Staleness

Staleness is not supported in theMonarch backend.

Calculation ofirate

When the lookback window for theirate functionis less than the step size, we increase the window to the step size.Monarch requires this change to ensure that none of the inputdata is completely ignored in the output. This difference applies torate calculations as well.

Calculation ofrate andincrease

When the lookback window for therate functionis less than the step size, we increase the window to the step size.Monarch requires this change to ensure that none of the input datais completely ignored in the output. This difference applies toirate calculations as well.

There are differences in the interpolation and extrapolation calculations.Monarch uses a different interpolation algorithm thanPrometheus, and this difference can lead to slightly different results.For example, Monarch counter samples are stored with a timerange rather than the single timestamp that Prometheus uses. Therefore,counter samples in Monarch can be included in a rate calculationeven though the Prometheus timestamp would exclude them. This generally resultsin more accurate rate results, especially when querying over the beginningor end of the underlying time series.

Calculation ofhistogram_quantile

A PromQLhistogram_quantilecalculation on a histogram with no samples produces a NaN value. The internalquery language's calculation produces no value;the point at the timestamp is dropped instead.

The rate-calculation differences can also affect the input tohistogram_quantile queries.

Type-specific functions ondifferently typed metrics

Although upstream Prometheus is weakly typed, Monarch is stronglytyped. This means that running functions specific to a single type on adifferently typed metric (for example, runningrate() on a GAUGE metric orhistogram_quantile() on a COUNTER or untyped metric) doesn't work inManaged Service for Prometheus, even though these functions work in upstreamPrometheus.