MetricsQL Syntax Reference¶

In the dashboards of TrueWatch, you can use the MetricsQL language to query Prometheus metrics.

MetricsQL is a PromQL syntax enhancement language developed by VictoriaMetrics. In most cases, MetricsQL is compatible with PromQL queries, but it also enhances and optimizes some aspects of PromQL that are less intuitive or convenient. This article introduces more details about the compatibility design.

In the vast majority of cases, you can follow the PromQL syntax you understand to make queries, and it usually works fine. When you encounter some more challenging problems, such as wanting to calculate the p95 of gauge data, or wanting topk to return the exact number of series, you can look for answers in the syntax reference below.

Comparing MetricsQL and PromQL syntax, the main differences are:

MetricsQL considers the previous point in the lookbehind window, which applies to range functions, such as rate and increase. This allows returning the exact result users expect for queries like increase(metric[$__interval]), rather than the incomplete results returned by Prometheus for such queries.
MetricsQL does not extrapolate the results of range functions, which addresses this issue in PromQL. For technical details on how rate and increase are calculated in PromQL and MetricsQL, see the comments in this issue.
MetricsQL returns the expected non-empty response for rate when the step value is less than the scrape interval. This addresses the problem displayed in Grafana. See also this blog post.
MetricsQL treats the scalar type the same as an instant vector without labels, because the subtle differences between these two types often confuse users. For details, see the corresponding Prometheus documentation.
MetricsQL removes all NaN values from the output, so some queries (e.g., (-1)^0.5) return empty results in MetricsQL, while they return a series of NaN values in PromQL. Note that the frontend does not draw any lines or points for NaN values, so the final result is the same whether using MetricsQL or PromQL.
MetricsQL preserves the metric name after applying functions that do not change the meaning of the original time series. For example, min_over_time(foo) or round(foo) preserves the foo metric name in the result. For details, see this issue.

At the same time, you should also note that when used in TrueWatch, there are still differences from the original MetricsQL:

Support for measurement selection: the measurement must be prepended to the metric name, separated by a colon, e.g., increase(measurement:metric[1m]). When the measurement is omitted, query performance degrades significantly.
The TrueWatch UI does not currently support manual configuration of step. The current step is automatically calculated based on the time range and display density.
The TrueWatch UI does not currently support Heatmap-type chart displays, which may affect the display of Histograms.

This article is primarily a translation of https://docs.victoriametrics.com/MetricsQL.html. If there are any ambiguities in the Chinese translation, you can also refer to the original text.

Feature List¶

MetricsQL includes the following features:

The lookbehind window in square brackets can be omitted. MetricsQL automatically selects the lookbehind window based on the step used to build the graph. The following query is valid in MetricsQL: rate(node_network_receive_bytes_total). When used in Grafana, it is equivalent to rate(node_network_receive_bytes_total[$__interval]).
Aggregate functions can accept any number of arguments. For example, avg(q1, q2, q3) returns the average of each point for q1, q2, and q3.
The @ modifier can be placed anywhere in the query. For example, sum(foo) @ end() calculates the value of sum(foo) at the end timestamp for the selected time range [start ... end].
Any subexpression can be used as the @ modifier. For example, foo @ (end() - 1h) calculates foo at the timestamp end - 1h for the selected time range [start ... end].
The offset, lookbehind window in square brackets, and step value in subqueries can reference the current step using the [Ni] syntax, also known as the $__interval value in Grafana. For example, rate(metric[10i] offset 5i) returns the per-second rate covering the previous 10 step values with a 5 step offset.
offset can be placed anywhere in the query. For example, sum(foo) offset 24h.
The lookbehind window in square brackets and offset can be fractional. For example, rate(node_network_receive_bytes_total[1.5m] offset 0.5d).
Duration suffixes are optional. If the suffix is omitted, the duration is in seconds. For example, rate(m[300] offset 1800) is equivalent to rate(m[5m]) offset 30m.
Duration can be placed anywhere in the query. For example, sum_over_time(m[1h]) / 1h is equivalent to sum_over_time(m[1h]) / 3600.
Numeric values can have K, Ki, M, Mi, G, Gi, T, and Ti suffixes. For example, 8K is equivalent to 8000, and 1.2Mi is equivalent to 1.2*1024*1024.
Trailing commas are allowed on all lists, such as label filters, function arguments, and expressions. For example, the following queries are valid: m{foo="bar",}, f(a, b,), WITH (x=y,) x, which simplifies the maintenance of multi-line queries.
Metric names and label names can contain any Unicode letters. For example, температура{город="Киев"} is a MetricsQL expression.
Metric names and label names can contain escape characters. For example, foo\\-bar{baz\\=aa="b"} is a valid expression. It returns a time series with the name foo-bar and a label baz=aa with value b. Additionally, the following escape sequences are supported:
- \\xXX, where XX is the hexadecimal representation of the escaped ASCII character.
- \\uXXXX, where XXXX is the hexadecimal representation of the escaped Unicode character.
Aggregate functions support an optional limit N suffix to limit the number of output series. For example, sum(x) by (y) limit 3 limits the number of output time series after aggregation to 3. All other time series are dropped.
histogram_quantile accepts an optional third argument boundsLabel. In this case, it returns the lower and upper bounds of the estimated percentile. For details, see this issue.
default binary operator. q1 default q2 fills gaps in q1 with the corresponding values from q2.
if binary operator. q1 if q2 drops values in q1 that are missing in q2.
ifnot binary operator. q1 ifnot q2 drops values in q1 that are present in q2.
WITH templates. This feature simplifies writing and managing complex queries. You can try it in the WITH templates playground.
String literals can be concatenated. This is useful in WITH templates: WITH (commonPrefix="long_metric_prefix_") {__name__=commonPrefix+"suffix1"} / {__name__=commonPrefix+"suffix2"}.
The keep_metric_names modifier can be applied to all Rollup functions and Transform functions. This modifier prevents the removal of metric names in function results. See these docs.

keep_metric_names¶

By default, metric names are removed after applying functions that change the meaning of the original time series. This can lead to "duplicate time series" errors when applying functions to multiple time series with different names. This error can be resolved by applying the keep_metric_names modifier to the function.

For example, rate({__name__=~"foo|bar"}) keep_metric_names preserves the metric names foo and bar in the returned time series.

MetricsQL Functions¶

MetricsQL provides the following functions:

Rollup functions
Transform functions
Label manipulation functions
Aggregate functions

Rollup Functions¶

Rollup functions (also known as range functions or window functions) perform rolling calculations on raw samples over a given lookbehind window, applicable to series selectors. For example, avg_over_time(temperature[24h]) calculates the average temperature of raw samples over the last 24 hours.

Additional details:

If a Rollup function is used to build a graph, each point on the graph is calculated independently. For example, each point of the avg_over_time(temperature[24h]) graph shows the average temperature over the last 24 hours. The interval between points is set by the step query parameter passed from the frontend.
If the given series selector returns multiple time series, the rolling calculation is performed separately for each returned series.
If the lookbehind window is missing in square brackets, MetricsQL automatically sets the lookbehind window to the interval between points on the graph (also known as the step query parameter in /api/v1/query_range, the $__interval value in Grafana, or the 1i duration in MetricsQL). For example, rate(http_requests_total) is equivalent to rate(http_requests_total[$__interval]); in MetricsQL, it is also equivalent to rate(http_requests_total[1i]).
Every series selector in MetricsQL must be wrapped in a Rollup function. Otherwise, it is automatically converted to default_rollup before calculation. For example, foo{bar="baz"} is automatically converted to default_rollup(foo{bar="baz"}[1i]) before calculation.
If something other than a series selector is passed to a Rollup function, the inner argument is automatically converted to a subquery.
All Rollup functions accept the optional keep_metric_names modifier. If set, the function preserves the metric name in the result. See these docs.

See implicit query conversions.

Supported Rollup Functions List¶

absent_over_time¶

absent_over_time(series_selector[d]) is a Rollup function that returns 1 if the given lookbehind window d does not contain any raw samples. Otherwise, it returns an empty result.

This function is supported by PromQL. See also present_over_time.

aggr_over_time¶

aggr_over_time(("rollup_func1","rollup_func2",...), series_selector[d]) is a Rollup function that calculates all the listed rollup_func* separately for each time series from the given series selector. rollup_func* can include any Rollup function. For example, aggr_over_time(("min_over_time","max_over_time","rate"), m[d]) will calculate min_over_time, max_over_time, and rate for m[d].

ascent_over_time¶

ascent_over_time(series_selector[d]) is a Rollup function that calculates the increase in raw sample values over the given lookbehind window d. The calculation is performed separately for each time series returned.

This function is useful for tracking altitude gain in GPS tracking. The metric name is stripped from the result. Add the keep_metric_names modifier to preserve the metric name.

See also descent_over_time.

avg_over_time¶

avg_over_time(series_selector[d]) is a Rollup function that calculates the average of raw sample values over the given lookbehind window d for each time series returned by the given series selector.

This function is supported by PromQL. See also median_over_time.

changes¶

changes(series_selector[d]) is a Rollup function that calculates the number of changes in raw sample values over the given lookbehind window d for each time series returned by the given series selector. Unlike Prometheus changes(), it considers the change from the last sample before the given lookbehind window d. For details, see this article.

The metric name is stripped from the result. Add the keep_metric_names modifier to preserve the metric name.

This function is supported by PromQL. See also changes_prometheus.

changes_prometheus¶

changes_prometheus(series_selector[d]) is a Rollup function that calculates the number of changes in raw sample values over the given lookbehind window d for each time series returned by the given series selector. It does not consider the change from the last sample before the given lookbehind window d, unlike Prometheus. For details, see this article.

The metric name is stripped from the result. Add the keep_metric_names modifier to preserve the metric name.

This function is supported by PromQL. See also changes.

count_eq_over_time¶

count_eq_over_time(series_selector[d], eq) is a Rollup function that counts the number of raw samples equal to eq over the given lookbehind window d for each time series returned by the given series selector. The metric name is stripped from the result. Add the keep_metric_names modifier to preserve the metric name.

See also count_over_time.

count_gt_over_time¶

count_gt_over_time(series_selector[d], gt) is a Rollup function that counts the number of raw samples greater than gt over the given lookbehind window d for each time series returned by the given series selector. The metric name is stripped from the result. Add the keep_metric_names modifier to preserve the metric name.

See also count_over_time.

count_le_over_time¶

count_le_over_time(series_selector[d], le) is a Rollup function that counts the number of raw samples less than or equal to le over the given lookbehind window d for each time series returned by the given series selector. The metric name is stripped from the result. Add the keep_metric_names modifier to preserve the metric name.

See also count_over_time.

count_ne_over_time¶

count_ne_over_time(series_selector[d], ne) is a Rollup function that counts the number of raw samples not equal to ne over the given lookbehind window d for each time series returned by the given series selector. The metric name is stripped from the result. Add the keep_metric_names modifier to preserve the metric name.

See also count_over_time.

count_over_time¶

count_over_time(series_selector[d]) is a Rollup function that counts the number of raw samples over the given lookbehind window d for each time series returned by the given series selector. The metric name is stripped from the result. Add the keep_metric_names modifier to preserve the metric name.

This function is supported by PromQL. See also count_le_over_time, count_gt_over_time, count_eq_over_time, and count_ne_over_time.

decreases_over_time¶

decreases_over_time(series_selector[d]) is a Rollup function that counts the number of decreases in raw sample values over the given lookbehind window d for each time series returned by the given series selector. The metric name is stripped from the result. Add the keep_metric_names modifier to preserve the metric name.

See also increases_over_time.

default_rollup¶

default_rollup(series_selector[d]) is a Rollup function that returns the last raw sample value over the given lookbehind window d for each time series returned by the given series selector.

delta¶

delta(series_selector[d]) is a Rollup function that calculates the difference between the last sample in the given lookbehind window d and the last sample before the given lookbehind window d for each time series returned by the given series selector. The behavior of delta() in MetricsQL differs slightly from that in Prometheus. For details, see this article.

The metric name is stripped from the result. Add the keep_metric_names modifier to preserve the metric name.

This function is supported by PromQL. See also increase and delta_prometheus.

delta_prometheus¶

delta_prometheus(series_selector[d]) is a Rollup function that calculates the difference between the first and last sample in the given lookbehind window d for each time series returned by the given series selector.

The behavior of delta_prometheus() is close to that of the delta() function in Prometheus. For details, see this article.

The metric name is stripped from the result. Add the keep_metric_names modifier to preserve the metric name.

See also delta.

deriv¶

deriv(series_selector[d]) is a Rollup function that calculates the per-second derivative of each time series returned by the given series selector, using linear regression.

The metric name is stripped from the result. Add the keep_metric_names modifier to preserve the metric name.

This function is supported by PromQL. See also deriv_fast and ideriv.

deriv_fast¶

deriv_fast(series_selector[d]) is a Rollup function that calculates the per-second derivative of each time series using the first and last raw samples in the given lookbehind window d.

The metric name is stripped from the result. Add the keep_metric_names modifier to preserve the metric name.

See also deriv and ideriv.

descent_over_time¶

descent_over_time(series_selector[d]) is a Rollup function that calculates the decrease in raw sample values over the given lookbehind window d. The calculation is performed separately for each time series returned.

This function is useful for tracking altitude loss in GPS tracking.

The metric name is stripped from the result. Add the keep_metric_names modifier to preserve the metric name.

See also ascent_over_time.

distinct_over_time¶

distinct_over_time(series_selector[d]) is a Rollup function that returns the number of distinct raw sample values for each time series over the given lookbehind window d.

The metric name is stripped from the result. Add the keep_metric_names modifier to preserve the metric name.

duration_over_time¶

duration_over_time(series_selector[d], max_interval) is a Rollup function that returns the duration (in seconds) the given series selector exists in the lookbehind window d. It is expected that the interval between adjacent samples in each series does not exceed max_interval. Otherwise, such intervals are considered gaps and are not counted.

The metric name is stripped from the result. Add the keep_metric_names modifier to preserve the metric name.

See also lifetime and lag.

first_over_time¶

first_over_time(series_selector[d]) is a Rollup function that returns the first raw sample value for each time series over the given lookbehind window d.

See also last_over_time and tfirst_over_time.

geomean_over_time¶

geomean_over_time(series_selector[d]) is a Rollup function that calculates the geometric mean of raw samples over the given lookbehind window d. The calculation is performed separately for each time series returned.

The metric name is stripped from the result. Add the keep_metric_names modifier to preserve the metric name.

histogram_over_time¶

histogram_over_time(series_selector[d]) is a Rollup function that computes a VictoriaMetrics histogram over raw samples in the given lookbehind window d. It is calculated separately for each time series from the given series selector. The resulting histograms are useful for passing to histogram_quantile to calculate percentiles from multiple gauges. For example, the following query calculates the median temperature per country over the last 24 hours:

histogram_quantile(0.5, sum(histogram_over_time(temperature[24h])) by (vmrange,country)).

hoeffding_bound_lower¶

hoeffding_bound_lower(phi, series_selector[d]) is a Rollup function that calculates the lower Hoeffding bound for the given phi in the range [0...1].

See also hoeffding_bound_upper.

hoeffding_bound_upper¶

hoeffding_bound_upper(phi, series_selector[d]) is a Rollup function that calculates the upper Hoeffding bound for the given phi in the range [0...1].

See also hoeffding_bound_lower.

holt_winters¶

holt_winters(series_selector[d], sf, tf) is a Rollup function that calculates the Holt-Winters value (also known as double exponential smoothing) of raw samples over the given lookbehind window d, using the given smoothing factor sf and trend factor tf. Both sf and tf must be in the range [0...1]. It is expected that the series selector returns time series of gauge type.

This function is supported by PromQL. See also range_linear_regression.

idelta¶

idelta(series_selector[d]) is a Rollup function that calculates the difference between the last two raw samples in each time series over the given lookbehind window d.

The metric name is stripped from the result. Add the keep_metric_names modifier to preserve the metric name.

This function is supported by PromQL. See also delta.

ideriv¶

ideriv(series_selector[d]) is a Rollup function that calculates the per-second derivative of each time series based on the last two raw samples over the given lookbehind window d. The calculation is performed separately for each time series from the given series selector.

The metric name is stripped from the result. Add the keep_metric_names modifier to preserve the metric name.

See also deriv.

increase¶

increase(series_selector[d]) is a Rollup function that calculates the increase over the given lookbehind window d for each time series. It is expected that the series selector returns time series of counter type.

Unlike Prometheus, it considers the last sample before the given lookbehind window d when calculating the result. For details, see this article.

The metric name is stripped from the result. Add the keep_metric_names modifier to preserve the metric name.

This function is supported by PromQL. See also increase_pure, increase_prometheus, and delta.

increase_prometheus¶

increase_prometheus(series_selector[d]) is a Rollup function that calculates the increase over the given lookbehind window d for each time series. It is expected that the series selector returns time series of counter type. It does not consider the last sample before the given lookbehind window d when calculating the result, unlike Prometheus. For details, see this article.

The metric name is stripped from the result. Add the keep_metric_names modifier to preserve the metric name.

See also increase_pure and increase.

increase_pure¶

increase_pure(series_selector[d]) is a Rollup function that is the same as increase, except that it assumes counters always start from 0, while increase ignores the first value if it is too large.

increases_over_time¶

increases_over_time(series_selector[d]) is a Rollup function that counts the number of increases in raw sample values over the given lookbehind window d for each time series.

The metric name is stripped from the result. Add the keep_metric_names modifier to preserve the metric name.

See also decreases_over_time.

integrate¶

integrate(series_selector[d]) is a Rollup function that calculates the integral of raw samples over the given lookbehind window d for each time series returned by the given series selector.

The metric name is stripped from the result. Add the keep_metric_names modifier to preserve the metric name.

irate¶

irate(series_selector[d]) is a Rollup function that calculates the instantaneous per-second growth rate of the last two raw samples over the given lookbehind window d for each time series returned by the given series selector. It is expected that series_selector returns time series of counter type.

The metric name is stripped from the result. Add the keep_metric_names modifier to preserve the metric name.

This function is supported by PromQL. See also rate and rollup_rate.

lag¶

lag(series_selector[d]) is a Rollup function that returns the duration (in seconds) between the last sample and the current point over the given lookbehind window d. It is calculated separately for each time series returned by the given series selector.

The metric name is stripped from the result. Add the keep_metric_names modifier to preserve the metric name.

See also lifetime and duration_over_time.

last_over_time¶

last_over_time(series_selector[d]) is a Rollup function that returns the last raw sample value for each time series returned by the given series selector over the given lookbehind window d.

This function is supported by PromQL. See also first_over_time and tlast_over_time.

lifetime¶

lifetime(series_selector[d]) is a Rollup function that returns the duration (in seconds) between the last and first sample over the given lookbehind window d, calculated separately for each time series returned by the given series selector.

The metric name is stripped from the result. Add the keep_metric_names modifier to preserve the metric name.

See also duration_over_time and lag.

mad_over_time¶

mad_over_time(series_selector[d]) is a Rollup function that calculates the median absolute deviation of raw samples over the given lookbehind window d for each time series returned by the given series selector.

See also mad and range_mad.

max_over_time¶

max_over_time(series_selector[d]) is a Rollup function that calculates the maximum of raw sample values over the given lookbehind window d for each time series returned by the given series selector.

This function is supported by PromQL. See also tmax_over_time.

median_over_time¶

median_over_time(series_selector[d]) is a Rollup function that calculates the median of raw sample values over the given lookbehind window d for each time series returned by the given series selector.

See also avg_over_time.

min_over_time¶

min_over_time(series_selector[d]) is a Rollup function that calculates the minimum of raw sample values over the given lookbehind window d for each time series returned by the given series selector.

This function is supported by PromQL. See also tmin_over_time.

mode_over_time¶

mode_over_time(series_selector[d]) is a Rollup function that calculates the mode of raw samples over the given lookbehind window d. It is calculated separately for each time series returned by the given series selector. Raw sample values are expected to be discrete.

predict_linear¶

predict_linear(series_selector[d], t) is a Rollup function that predicts the value t seconds from now, using linear interpolation of raw sample values over the given lookbehind window d. The prediction is calculated separately for each time series returned by the given series selector.

This function is supported by PromQL. See also range_linear_regression.

present_over_time¶

present_over_time(series_selector[d]) is a Rollup function that returns 1 if at least one raw sample exists over the given lookbehind window d. Otherwise, it returns an empty result.

The metric name is stripped from the result. Add the keep_metric_names modifier to preserve the metric name.

This function is supported by PromQL.

quantile_over_time¶

quantile_over_time(phi, series_selector[d]) is a Rollup function that calculates the phi quantile over the given lookbehind window d for each time series returned by the given series selector. The phi value must be in the range [0...1].

This function is supported by PromQL. See also quantiles_over_time.

quantiles_over_time¶

quantiles_over_time("phiLabel", phi1, ..., phiN, series_selector[d]) is a Rollup function that calculates the phi* quantiles over the given lookbehind window d for each time series returned by the given series selector. The function returns a separate time series for each phi*, with the {phiLabel="phi*"} label. The phi* values must be in the range [0...1].

See also quantile_over_time.

range_over_time¶

range_over_time(series_selector[d]) is a Rollup function that calculates the range of values over the given lookbehind window d for each time series returned by the given series selector. For example, it calculates max_over_time(series_selector[d]) - min_over_time(series_selector[d]).

The metric name is stripped from the result. Add the keep_metric_names modifier to preserve the metric name.

rate¶

rate(series_selector[d]) is a Rollup function that calculates the average per-second growth rate over the given lookbehind window d for each time series returned by the given series selector. It is expected that series_selector returns time series of counter type.

The metric name is stripped from the result. Add the keep_metric_names modifier to preserve the metric name.

This function is supported by PromQL. See also irate and rollup_rate.

rate_over_sum¶

rate_over_sum(series_selector[d]) is a Rollup function that calculates the per-second rate of the sum of raw sample values over the given lookbehind window d for each time series returned by the given series selector. The calculations are performed separately for each time series.

The metric name is stripped from the result. Add the keep_metric_names modifier to preserve the metric name.

resets¶

resets(series_selector[d]) is a Rollup function that returns the number of counter resets over the given lookbehind window d for each time series returned by the given series selector. It is expected that series_selector returns time series of counter type.

The metric name is stripped from the result. Add the keep_metric_names modifier to preserve the metric name.

This function is supported by PromQL.

rollup¶

rollup(series_selector[d]) is a Rollup function that calculates the min, max, and avg values of raw samples over the given lookbehind window d, and returns them in time series with additional labels rollup="min", rollup="max", and rollup="avg". These values are calculated separately for each time series returned by the given series selector.

An optional second argument "min", "max", or "avg" can be passed to keep only one calculation result and not add labels.

rollup_candlestick¶

rollup_candlestick(series_selector[d]) is a Rollup function that calculates the open, high, low, and close values (also known as OHLC) over the given lookbehind window d and returns them as time series with additional labels rollup="open", rollup="high", rollup="low", and rollup="close". These calculations are performed separately for each time series returned by the given series selector. This function is useful for financial applications.

An optional second argument "min", "max", or "avg" can be passed to keep only one calculation result and not add labels.

rollup_delta¶

rollup_delta(series_selector[d]) is a Rollup function that calculates the difference between adjacent raw samples over the given lookbehind window d, and returns the min, max, and avg values of the calculated differences as time series with additional labels rollup="min", rollup="max", and rollup="avg". These calculations are performed separately for each time series returned by the given series selector.

An optional second argument "min", "max", or "avg" can be passed to keep only one calculation result and not add labels.

The metric name is stripped from the result. Add the keep_metric_names modifier to preserve the metric name.

See also rollup_increase.

rollup_deriv¶

rollup_deriv(series_selector[d]) is a Rollup function that calculates the per-second derivative of adjacent raw samples over the given lookbehind window d, and returns the min, max, and avg values of the calculated per-second derivatives as time series with additional labels rollup="min", rollup="max", and rollup="avg". These calculations are performed separately for each time series returned by the given series selector.

An optional second argument "min", "max", or "avg" can be passed to keep only one calculation result and not add labels.

The metric name is stripped from the result. Add the keep_metric_names modifier to preserve the metric name.

rollup_increase¶

rollup_increase(series_selector[d]) is a Rollup function that calculates the increase between adjacent raw samples over the given lookbehind window d, and returns the min, max, and avg values of the calculated increases as time series with additional labels rollup="min", rollup="max", and rollup="avg". These calculations are performed separately for each time series returned by the given series selector.

An optional second argument "min", "max", or "avg" can be passed to keep only one calculation result and not add labels.

The metric name is stripped from the result. Add the keep_metric_names modifier to preserve the metric name. See also rollup_delta.

rollup_rate¶

rollup_rate(series_selector[d]) is a Rollup function that calculates the per-second change rate of adjacent raw samples over the given lookbehind window d, and returns the min, max, and avg values of the calculated per-second change rates as time series with additional labels rollup="min", rollup="max", and rollup="avg".

See this article for a better understanding of when to use rollup_rate().

An optional second argument "min", "max", or "avg" can be passed to keep only one calculation result and not add labels.

These calculations are performed separately for each time series from the given series selector.

The metric name is stripped from the result. Add the keep_metric_names modifier to preserve the metric name.

rollup_scrape_interval¶

rollup_scrape_interval(series_selector[d]) is a Rollup function that calculates the interval (in seconds) between adjacent raw samples over the given lookbehind window d, and returns the min, max, and avg values of the calculated intervals as time series with additional labels rollup="min", rollup="max", and rollup="avg".

These calculations are performed separately for each time series from the given series selector.

An optional second argument "min", "max", or "avg" can be passed to keep only one calculation result and not add labels.

The metric name is stripped from the result. Add the keep_metric_names modifier to preserve the metric name. See also scrape_interval.

scrape_interval¶

scrape_interval(series_selector[d]) is a Rollup function that calculates the average interval (in seconds) between raw samples over the given lookbehind window d, for each time series from the given series selector.

The metric name is stripped from the result. Add the keep_metric_names modifier to preserve the metric name.

See also rollup_scrape_interval.

share_gt_over_time¶

share_gt_over_time(series_selector[d], gt) is a Rollup function that returns the share (in the interval [0...1]) of raw samples greater than gt over the given lookbehind window d. It is calculated independently for each time series from the given series selector.

This function is useful for calculating SLI and SLO. For example: share_gt_over_time(up[24h], 0) - returns the service availability over the last 24 hours.

The metric name is stripped from the result. Add the keep_metric_names modifier to preserve the metric name.

See also share_le_over_time.

share_le_over_time¶

share_le_over_time(series_selector[d], le) is a Rollup function that returns the share (in the interval [0...1]) of raw samples less than or equal to le over the given lookbehind window d. It is calculated independently for each time series from the given series selector.

This function is useful for calculating SLI and SLO. For example: share_le_over_time(memory_usage_bytes[24h], 100*1024*1024) returns the share of time series values with memory usage less than or equal to 100MB over the last 24 hours.

The metric name is stripped from the result. Add the keep_metric_names modifier to preserve the metric name.

See also share_gt_over_time.

stale_samples_over_time¶

stale_samples_over_time(series_selector[d]) is a Rollup function that counts the number of staleness markers for each time series matching the given series selector over the given lookbehind window d.

The metric name is stripped from the result. Add the keep_metric_names modifier to preserve the metric name.

stddev_over_time¶

stddev_over_time(series_selector[d]) is a Rollup function that calculates the standard deviation of raw samples over the given lookbehind window d for each time series from the given series selector.

The metric name is stripped from the result. Add the keep_metric_names modifier to preserve the metric name.

This function is supported by PromQL. See also stdvar_over_time.

stdvar_over_time¶

stdvar_over_time(series_selector[d]) is a Rollup function that calculates the standard variance of raw samples over the given lookbehind window d for each time series from the given series selector.

The metric name is stripped from the result. Add the keep_metric_names modifier to preserve the metric name.

This function is supported by PromQL. See also stddev_over_time.

sum_over_time¶

sum_over_time(series_selector[d]) is a Rollup function that calculates the sum of raw sample values over the given lookbehind window d for each time series from the given series selector.

The metric name is stripped from the result. Add the keep_metric_names modifier to preserve the metric name.

This function is supported by PromQL.

sum2_over_time¶

sum2_over_time(series_selector[d]) is a Rollup function that calculates the sum of squares of raw sample values over the given lookbehind window d for each time series from the given series selector.

The metric name is stripped from the result. Add the keep_metric_names modifier to preserve the metric name.

timestamp¶

timestamp(series_selector[d]) is a Rollup function that returns the timestamp (in seconds) of the last raw sample for each time series from the given series selector over the given lookbehind window d.

The metric name is stripped from the result. Add the keep_metric_names modifier to preserve the metric name.

This function is supported by PromQL. See also timestamp_with_name.

timestamp_with_name¶

timestamp_with_name(series_selector[d]) is a Rollup function that returns the timestamp (in seconds) of the last raw sample for each time series from the given series selector over the given lookbehind window d.

The metric name is preserved in the result.

See also timestamp.

tfirst_over_time¶

tfirst_over_time(series_selector[d]) is a Rollup function that returns the timestamp (in seconds) of the first raw sample for each time series from the given series selector over the given lookbehind window d.

The metric name is stripped from the result. Add the keep_metric_names modifier to preserve the metric name.

See also first_over_time.

tlast_change_over_time¶

tlast_change_over_time(series_selector[d]) is a Rollup function that returns the timestamp of the last change for each time series from the given series selector over the given lookbehind window d.

The metric name is stripped from the result. Add the keep_metric_names modifier to preserve the metric name.

See also last_over_time.

tlast_over_time¶

tlast_over_time(series_selector[d]) is a Rollup function that is an alias of timestamp.

See also tlast_change_over_time.

tmax_over_time¶

tmax_over_time(series_selector[d]) is a Rollup function that returns the timestamp (in seconds) of the raw sample with the maximum value over the given lookbehind window d. It is calculated independently for each time series from the given series selector.

The metric name is stripped from the result. Add the keep_metric_names modifier to preserve the metric name.

See also max_over_time.

tmin_over_time¶

tmin_over_time(series_selector[d]) is a Rollup function that returns the timestamp (in seconds) of the raw sample with the minimum value over the given lookbehind window d. It is calculated independently for each time series from the given series selector.

The metric name is stripped from the result. Add the keep_metric_names modifier to preserve the metric name.

See also min_over_time.

zscore_over_time¶

zscore_over_time(series_selector[d]) is a Rollup function that returns the z-score of raw samples over the given lookbehind window d. It is calculated independently for each time series from the given series selector.

The metric name is stripped from the result. Add the keep_metric_names modifier to preserve the metric name.

See also zscore and range_trim_zscore.

Transform Functions¶

Transform functions compute transformations of Rollup results. For example, abs(delta(temperature[24h])) calculates the absolute value of each point for each time series returned from the rollup.

Additional details:

If a transform function is applied directly to a series selector, the default_rollup() function is automatically applied before the transformation. For example, abs(temperature) is implicitly converted to abs(default_rollup(temperature[1i])).
All transform functions accept the optional keep_metric_names modifier. If set, the function does not remove the metric name from the resulting time series. See these docs.

See also implicit query conversions.

Supported Transform Functions List¶

abs¶

abs(q) is a Transform function that calculates the absolute value of each point for each time series returned by q.

This function is supported by PromQL.

absent¶

absent(q) is a Transform function that returns 1 if q has no points. Otherwise, it returns an empty result.

This function is supported by PromQL. See also absent_over_time.

acos¶

acos(q) is a Transform function that returns the arccosine of each point for each time series returned by q.

The metric name is stripped from the result. Add the keep_metric_names modifier to preserve the metric name.

This function is supported by PromQL. See also asin and cos.

acosh¶

acosh(q) is a Transform function that returns the inverse hyperbolic cosine of each point for each time series returned by q.

The metric name is stripped from the result. Add the keep_metric_names modifier to preserve the metric name.

This function is supported by PromQL. See also sinh.

asin¶

asin(q) is a Transform function that returns the arcsine of each point for each time series returned by q.

The metric name is stripped from the result. Add the keep_metric_names modifier to preserve the metric name.

This function is supported by PromQL. See also acos and sin.

asinh¶

asinh(q) is a Transform function that returns the inverse hyperbolic sine of each point for each time series returned by q.

The metric name is stripped from the result. Add the keep_metric_names modifier to preserve the metric name.

This function is supported by PromQL. See also sinh.

atan¶

atan(q) is a Transform function that returns the arctangent of each point for each time series returned by q.

The metric name is stripped from the result. Add the keep_metric_names modifier to preserve the metric name.

This function is supported by PromQL. See also tan.

atanh¶

atanh(q) is a Transform function that returns the inverse hyperbolic tangent of each point for each time series returned by q.

The metric name is stripped from the result. Add the keep_metric_names modifier to preserve the metric name.

This function is supported by PromQL. See also tanh.

bitmap_and¶

bitmap_and(q, mask) is a Transform function that calculates the bitwise v & mask for each point v of each time series returned by q.

The metric name is stripped from the result. Add the keep_metric_names modifier to preserve the metric name.

bitmap_or¶

bitmap_or(q, mask) is a Transform function that calculates the bitwise v | mask for each point v of each time series returned by q.

The metric name is stripped from the result. Add the keep_metric_names modifier to preserve the metric name.

bitmap_xor¶

bitmap_xor(q, mask) is a Transform function that calculates the bitwise v ^ mask for each point v of each time series returned by q.

The metric name is stripped from the result. Add the keep_metric_names modifier to preserve the metric name.

buckets_limit¶

buckets_limit(limit, buckets) is a Transform function that limits the number of histogram buckets to the given limit.

See also prometheus_buckets and histogram_quantile.

ceil¶

ceil(q) is a Transform function that rounds each point of each time series returned by q up to the nearest integer.

This function is supported by PromQL. See also floor and round.

clamp¶

clamp(q, min, max) is a Transform function that clamps each point of each time series returned by q to the given min and max values.

This function is supported by PromQL. See also clamp_min and clamp_max.

clamp_max¶

clamp_max(q, max) is a Transform function that clamps each point of each time series returned by q to the given max value.

This function is supported by PromQL. See clamp and clamp_min.

clamp_min¶

clamp_min(q, min) is a Transform function that clamps each point of each time series returned by q to the given min value.

This function is supported by PromQL. See clamp and clamp_max.

cos¶

cos(q) is a Transform function that returns cos(v) for each point v of each time series returned by q.

The metric name is stripped from the result. Add the keep_metric_names modifier to preserve the metric name.

This function is supported by PromQL. See sin.

cosh¶

cosh(q) is a Transform function that returns the hyperbolic cosine of each point of each time series returned by q.

The metric name is stripped from the result. Add the keep_metric_names modifier to preserve the metric name.

This function is supported by PromQL. See also acosh.

day_of_month¶

day_of_month(q) is a Transform function that returns the day of the month for each point of each time series returned by q. It is expected that q returns Unix timestamps. The returned value is in the range [1...31].

The metric name is stripped from the result. Add the keep_metric_names modifier to preserve the metric name.

This function is supported by PromQL.

day_of_week¶

day_of_week(q) is a Transform function that returns the day of the week for each point of each time series returned by q. It is expected that q returns Unix timestamps. The returned value is in the range [0...6], where 0 is Sunday and 6 is Saturday.

The metric name is stripped from the result. Add the keep_metric_names modifier to preserve the metric name.

This function is supported by PromQL.

days_in_month¶

days_in_month(q) is a Transform function that returns the number of days in the month identified by each point of each time series returned by q. It is expected that q returns Unix timestamps. The returned value is in the range [28...31].

The metric name is stripped from the result. Add the keep_metric_names modifier to preserve the metric name.

This function is supported by PromQL.

deg¶

deg(q) is a Transform function that converts each point of each time series returned by q from radians to degrees.

The metric name is stripped from the result. Add the keep_metric_names modifier to preserve the metric name.

This function is supported by PromQL. See also rad.

end¶

end() is a Transform function that returns the Unix timestamp (in seconds) of the last point. It is called the end query parameter passed to /api/v1/query_range.

See also start, time, and now.

exp¶

exp(q) is a Transform function that calculates e^v for each point of each time series returned by q.

The metric name is stripped from the result. Add the keep_metric_names modifier to preserve the metric name.

This function is supported by PromQL. See also ln.

floor¶

floor(q) is a Transform function that rounds each point of each time series returned by q down to the nearest integer.

This function is supported by PromQL. See also ceil and round.

histogram_avg¶

histogram_avg(buckets) is a Transform function that calculates the average of the given buckets. It can be used to calculate the average over a given time range across multiple time series. For example, histogram_avg(sum(histogram_over_time(response_time_duration_seconds[5m])) by (vmrange,job)) returns the average response time for each job over the last 5 minutes.

histogram_quantile¶

histogram_quantile(phi, buckets) is a Transform function that calculates the phi - percentile over the given histogram buckets. phi must be in the range [0...1]. For example, histogram_quantile(0.5, sum(rate(http_request_duration_seconds_bucket[5m]) by (le)) returns the median request duration over all requests in the last 5 minutes.

The function accepts an optional third argument - boundsLabel. In this case, it returns the lower and upper bounds of the estimated percentile with the given boundsLabel label. For details, see this issue.

When calculating percentiles over multiple histograms, all input histograms must have the same bucket boundaries, i.e., they must have the same set of le or vmrange labels. Otherwise, the returned result may be invalid. For details, see this issue.

This function is supported by PromQL (except for the boundLabel argument). See also histogram_quantiles, histogram_share, and quantile.

histogram_quantiles¶

histogram_quantiles("phiLabel", phi1, ..., phiN, buckets) is a Transform function that calculates the given phi* - quantiles over the given histogram buckets. The phi* arguments must be in the range [0...1]. For example, histogram_quantiles('le', 0.3, 0.5, sum(rate(http_request_duration_seconds_bucket[5m]) by (le)). Each calculated quantile is returned in a separate time series with the corresponding {phiLabel="phi*"} label.

See also histogram_quantile.

histogram_share¶

histogram_share(le, buckets) is a Transform function that calculates the share (in the range [0...1]) of points below le in the given buckets. This function is useful for calculating SLI and SLO. It is the inverse of histogram_quantile.

The function accepts an optional third argument - boundsLabel. In this case, it returns the lower and upper bounds of the estimated share with the given boundsLabel label.

histogram_stddev¶

histogram_stddev(buckets) is a Transform function that calculates the standard deviation of the given buckets.

histogram_stdvar¶

histogram_stdvar(buckets) is a Transform function that calculates the standard variance of the given buckets. It can be used to calculate the standard deviation over a given time range across multiple time series. For example, histogram_stdvar(sum(histogram_over_time(temperature[24])) by (vmrange,country)) returns the standard deviation of temperature for each country over the last 24 hours.

hour¶

hour(q) is a Transform function that returns the hour for each point of each time series returned by q. It is expected that q returns Unix timestamps. The returned value is in the range [0...23].

The metric name is stripped from the result. Add the keep_metric_names modifier to preserve the metric name.

This function is supported by PromQL.

interpolate¶

interpolate(q) is a Transform function that fills gaps with linearly interpolated values calculated from the previous and next non-null points of each time series returned by q.

See also keep_last_value and keep_next_value.

keep_last_value¶

keep_last_value(q) is a Transform function that fills gaps with the value of the last non-null point in each returned time series.

See also keep_next_value and interpolate.

keep_next_value¶

keep_next_value(q) is a Transform function that fills gaps with the value of the next non-null point in each returned time series.

See also keep_last_value and interpolate.

limit_offset¶

limit_offset(limit, offset, q) is a Transform function that skips the first offset time series from q and then returns at most limit remaining time series.

This allows simple pagination for q time series. See also limitk.

ln¶

ln(q) is a Transform function that calculates ln(v) for each point of each time series returned by q.

The metric name is stripped from the result. Add the keep_metric_names modifier to preserve the metric name.

This function is supported by PromQL. See also exp and log2.

log2¶

log2(q) is a Transform function that calculates log2(v) for each point of each time series returned by q.

The metric name is stripped from the result. Add the keep_metric_names modifier to preserve the metric name.

This function is supported by PromQL. See also log10 and ln.

log10¶

log10(q) is a Transform function that calculates log10(v) for each point of each time series returned by q.

The metric name is stripped from the result. Add the keep_metric_names modifier to preserve the metric name.

This function is supported by PromQL. See also log2 and ln.

minute¶

minute(q) is a Transform function that returns the minute for each point of each time series returned by q. It is expected that q returns Unix timestamps. The returned value is in the range [0...59].

The metric name is stripped from the result. Add the keep_metric_names modifier to preserve the metric name.

This function is supported by PromQL.

month¶

month(q) is a Transform function that returns the month for each point of each time series returned by q. It is expected that q returns Unix timestamps. The returned value is in the range [1...12], where 1 is January and 12 is December.

The metric name is stripped from the result. Add the keep_metric_names modifier to preserve the metric name.

This function is supported by PromQL.

now¶

now() is a Transform function that returns the current timestamp as a floating point value in seconds.

See also time.

pi¶

pi() is a Transform function that returns the Pi number.

This function is supported by PromQL.

rad¶

rad(q) is a Transform function that converts each point of each time series returned by q from degrees to radians.

The metric name is stripped from the result. Add the keep_metric_names modifier to preserve the metric name.

This function is supported by PromQL. See also deg.

prometheus_buckets¶

prometheus_buckets(buckets) is a Transform function that converts VictoriaMetrics histogram buckets with vmrange labels to Prometheus histogram buckets with le labels.

See also histogram_quantile and buckets_limit.

rand¶

rand(seed) is a Transform function that returns pseudo-random numbers uniformly distributed in the range [0...1]. An optional seed can be used as the seed for the pseudo-random number generator.

See also rand_normal and rand_exponential.

rand_exponential¶

rand_exponential(seed) is a Transform function that returns pseudo-random numbers with an exponential distribution. An optional seed can be used as the seed for the pseudo-random number generator.

See also rand and rand_normal.

rand_normal¶

rand_normal(seed) is a Transform function that returns pseudo-random numbers with a normal distribution. An optional seed can be used as the seed for the pseudo-random number generator.

See also rand and rand_exponential.

range_avg¶

range_avg(q) is a Transform function that calculates the average of points in each time series returned by q.

range_first¶

range_first(q) is a Transform function that returns the value of the first point in each time series returned by q.

range_last¶

range_last(q) is a Transform function that returns the value of the last point in each time series returned by q.

range_linear_regression¶

range_linear_regression(q) is a Transform function that calculates the simple linear regression for each time series returned by q over the selected time range. This function is useful for capacity planning and forecasting.

range_mad¶

range_mad(q) is a Transform function that calculates the median absolute deviation among points of each time series returned by q.

See also mad and mad_over_time.

range_max¶

range_max(q) is a Transform function that calculates the maximum among points of each time series returned by q.

range_median¶

range_median(q) is a Transform function that calculates the median among points of each time series returned by q.

range_min¶

range_min(q) is a Transform function that calculates the minimum among points of each time series returned by q.

range_normalize¶

range_normalize(q1, ...) is a Transform function that normalizes the values of time series returned by q1, ... to the [0 ... 1] range. This function is useful for correlating time series with different value ranges.

See also share.

range_quantile¶

range_quantile(phi, q) is a Transform function that returns the phi quantile from each time series returned by q. phi must be in the range [0 ... 1].

range_stddev¶

range_stddev(q) is a Transform function that calculates the standard deviation of each time series returned by q over the selected time range.

range_stdvar¶

range_stdvar(q) is a Transform function that calculates the standard variance of each time series returned by q over the selected time range.

range_sum¶

range_sum(q) is a Transform function that calculates the sum of points in each time series returned by q.

range_trim_outliers¶

range_trim_outliers(k, q) is a Transform function that removes points that are farther than k*range_mad(q) from range_median(q). For example, it is equivalent to the following query: q ifnot (abs(q - range_median(q)) > k*range_mad(q)).

See also range_trim_spikes and range_trim_zscore.

range_trim_spikes¶

range_trim_spikes(phi, q) is a Transform function that removes the largest phi percentage of spikes from the time series returned by q. phi must be in the range [0..1], where 0 means 0% and 1 means 100%.

See also range_trim_outliers and range_trim_zscore.

range_trim_zscore¶

range_trim_zscore(z, q) is a Transform function that removes points that are farther than z*range_stddev(q) from range_avg(q). For example, it is equivalent to the following query: q ifnot (abs(q - range_avg(q)) > z*range_avg(q)).

See also range_trim_outliers and range_trim_spikes.

range_zscore¶

range_zscore(q) is a Transform function that calculates the z-score for points returned by q. For example, it is equivalent to the following query: (q - range_avg(q)) / range_stddev(q).

remove_resets¶

remove_resets(q) is a Transform function that removes counter resets from time series returned by q.

round¶

round(q, nearest) is a Transform function that rounds each point of each time series returned by q to the nearest multiple of nearest. If nearest is omitted, it rounds to the nearest integer.

This function is supported by PromQL. See also floor and ceil.

ru¶

ru(free, max) is a Transform function that calculates resource utilization in the range [0% ... 100%] for the given free and max resources. For example, ru(node_memory_MemFree_bytes, node_memory_MemTotal_bytes) returns memory utilization for node_exporter metrics.

running_avg¶

running_avg(q) is a Transform function that calculates the running average of each time series returned by q.

running_max¶

running_max(q) is a Transform function that calculates the running maximum of each time series returned by q.

running_min¶

running_min(q) is a Transform function that calculates the running minimum of each time series returned by q.

running_sum¶

running_sum(q) is a Transform function that calculates the running sum of each time series returned by q.

scalar¶

scalar(q) is a Transform function that returns q if q contains only a single time series. Otherwise, it returns nothing.

This function is supported by PromQL.

sgn¶

sgn(q) is a Transform function that returns 1 if v > 0, -1 if v < 0, and 0 if v == 0 for each point v of each time series returned by q.

The metric name is stripped from the result. Add the keep_metric_names modifier to preserve the metric name.

This function is supported by PromQL.

sin¶

sin(q) is a Transform function that returns sin(v) for each point v of each time series returned by q.

The metric name is stripped from the result. Add the keep_metric_names modifier to preserve the metric name.

This function is supported by MetricsQL. See also cos.

sinh¶

sinh(q) is a Transform function that returns the hyperbolic sine of each point of each time series returned by q.

The metric name is stripped from the result. Add the keep_metric_names modifier to preserve the metric name.

This function is supported by MetricsQL. See also cosh.

tan¶

tan(q) is a Transform function that returns tan(v) for each point v of each time series returned by q.

The metric name is stripped from the result. Add the keep_metric_names modifier to preserve the metric name.

This function is supported by MetricsQL. See also atan.

tanh¶

tanh(q) is a Transform function that returns the hyperbolic tangent of each point of each time series returned by q.

The metric name is stripped from the result. Add the keep_metric_names modifier to preserve the metric name.

This function is supported by MetricsQL. See also atanh.

smooth_exponential¶

smooth_exponential(q, sf) is a Transform function that smooths points of each time series returned by q using the given smoothing factor sf.

sort¶

sort(q) is a Transform function that sorts series in ascending order by the last point in each time series returned by q.

This function is supported by PromQL. See also sort_desc and sort_by_label.

sort_desc¶

sort_desc(q) is a Transform function that sorts series in descending order by the last point of each time series returned by q.

This function is supported by PromQL. See also sort and sort_by_label.

sqrt¶

sqrt(q) is a Transform function that calculates the square root of each point of each time series returned by q.

The metric name is stripped from the result. Add the keep_metric_names modifier to preserve the metric name.

This function is supported by PromQL.

start¶

start() is a Transform function that returns the Unix timestamp (in seconds) of the first point.

It is called the start query parameter passed to /api/v1/query_range.

See also end, time, and now.

step¶

step() is a Transform function that returns the step (also known as interval) between points in seconds. It is called the step query parameter passed to /api/v1/query_range.

See also start and end.

time¶

time() is a Transform function that returns the Unix timestamp of each returned point.

This function is supported by PromQL. See also now, start, and end.

timezone_offset¶

timezone_offset(tz) is a Transform function that returns the offset in seconds relative to UTC for the given timezone tz. This can be useful when combined with date-time-related functions. For example, day_of_week(time()+timezone_offset("America/Los_Angeles")) returns the day of the week in the America/Los_Angeles timezone.

The special Local timezone can be used to return the offset of the timezone set on the host running VictoriaMetrics.

See the list of supported timezones.

ttf¶

ttf(free) is a Transform function that estimates the time (in seconds) required to exhaust the free resource. For example, ttf(node_filesystem_avail_byte) returns the time to exhaust storage space. This function may be useful for capacity planning.

union¶

union(q1, ..., qN) is a Transform function that returns the union of time series returned by q1, ..., qN. The union function name can be omitted - the following queries are equivalent: union(q1, q2) and (q1, q2).

It is expected that each q* query returns time series with unique label sets. Otherwise, only the first time series from the set with the same labels is returned. Use the alias and label_set functions to provide unique label sets for each q* query:

vector¶

vector(q) is a Transform function that returns q, i.e., it does nothing in MetricsQL.

This function is supported by PromQL.

year¶

year(q) is a Transform function that returns the year for each point of each time series returned by q. It is expected that q returns Unix timestamps.

The metric name is stripped from the result. Add the keep_metric_names modifier to preserve the metric name.

This function is supported by PromQL.

Label Manipulation Functions¶

Label manipulation functions are used to perform label manipulations on selected Rollup results.

Additional details:

If a label manipulation function is applied directly to a series selector, the default_rollup() function is automatically applied before the label transformation. For example, alias(temperature, "foo") is implicitly converted to alias(default_rollup(temperature[1i]), "foo").

See also implicit query conversions.

Supported Label Manipulation Functions List¶

alias¶

alias(q, "name") is a label manipulation function that sets the given name for all time series returned by q. For example, alias(up, "foobar") renames the up series to foobar series.

drop_common_labels¶

drop_common_labels(q1, ...., qN) is a label manipulation function that removes common label="value" pairs from the time series returned by q1, ..., qN.

label_copy¶

label_copy(q, "src_label1", "dst_label1", ..., "src_labelN", "dst_labelN") is a label manipulation function that copies the label values from src_label* to dst_label* for all time series returned by q. If src_label is empty, the corresponding dst_label remains unchanged.

label_del¶

label_del(q, "label1", ..., "labelN") is a label manipulation function that removes the given label* from all time series returned by q.

label_join¶

label_join(q, "dst_label", "separator", "src_label1", ..., "src_labelN") is a label manipulation function that concatenates the src_label* values with the given separator and stores the result in dst_label. This is performed separately for each time series returned by q. For example, label_join(up{instance="xxx",job="yyy"}, "foo", "-", "instance", "job") stores the xxx-yyy label value in the foo label.

This function is supported by PromQL.

label_keep¶

label_keep(q, "label1", ..., "labelN") is a label manipulation function that keeps only the listed label* labels in all time series returned by q and removes all others.

label_lowercase¶

label_lowercase(q, "label1", ..., "labelN") is a label manipulation function that converts the values of the given label* labels to lowercase for all time series returned by q.

label_map¶

label_map(q, "label", "src_value1", "dst_value1", ..., "src_valueN", "dst_valueN") is a label manipulation function that maps the label values from src_* to dst_* for all time series returned by q.

label_match¶

label_match(q, "label", "regexp") is a label manipulation function that removes time series from q where the label does not match the given regexp. This function is useful after functions like Rollup that may return multiple time series for each input series.

See also label_mismatch.

label_mismatch¶

label_mismatch(q, "label", "regexp") is a label manipulation function that removes time series from q where the label matches the given regexp. This function is useful after functions like Rollup that may return multiple time series for each input series.

See also label_match.

label_move¶

label_move(q, "src_label1", "dst_label1", ..., "src_labelN", "dst_labelN") is a label manipulation function that moves the label values from src_label* to dst_label* for all time series returned by q. If src_label is empty, the corresponding dst_label remains unchanged.

label_replace¶

label_replace(q, "dst_label", "replacement", "src_label", "regex") is a label manipulation function that applies the given regex to src_label and stores the replacement in dst_label when the given regex matches src_label. The replacement can contain references to regex capture groups, such as $1, $2, etc. These references are replaced by the corresponding regex captures. For example, label_replace(up{job="node-exporter"}, "foo", "bar-$1", "job", "node-(.+)") stores the bar-exporter label value in the foo label.

This function is supported by PromQL.

label_set¶

label_set(q, "label1", "value1", ..., "labelN", "valueN") is a label manipulation function that sets the {label1="value1", ..., labelN="valueN"} labels for all time series returned by q.

label_transform¶

label_transform(q, "label", "regexp", "replacement") is a label manipulation function that replaces all occurrences of regexp in the given label with the given replacement.

label_uppercase¶

label_uppercase(q, "label1", ..., "labelN") is a label manipulation function that converts the values of the given label* labels to uppercase for all time series returned by q.

See also label_lowercase.

label_value¶

label_value(q, "label") is a label manipulation function that obtains the numeric value of the given label for each time series returned by q.

For example, if label_value(foo, "bar") is applied to foo{bar="1.234"}, it returns a time series foo{bar="1.234"} with the value 1.234. For non-numeric label values, the function returns no data.

sort_by_label¶

sort_by_label(q, label1, ... labelN) is a label manipulation function that sorts series in ascending order by the given set of labels. For example, sort_by_label(foo, "bar") sorts the foo series by the values of the label bar in those series.

See also sort_by_label_desc and sort_by_label_numeric.

sort_by_label_desc¶

sort_by_label_desc(q, label1, ... labelN) is a label manipulation function that sorts series in descending order by the given set of labels. For example, sort_by_label(foo, "bar") sorts the foo series by the values of the label bar in those series.

See also sort_by_label and sort_by_label_numeric_desc.

sort_by_label_numeric¶

sort_by_label_numeric(q, label1, ... labelN) is a label manipulation function that sorts series in ascending order by the given set of labels using numeric sort. For example, if the foo series have bar label values of 1, 101, 15, and 2, sort_by_label_numeric(foo, "bar") returns the series in the order of bar label values: 1, 2, 15, and 101.

See also sort_by_label_numeric_desc and sort_by_label.

sort_by_label_numeric_desc¶

sort_by_label_numeric_desc(q, label1, ... labelN) is a label manipulation function that sorts series in descending order by the given set of labels using numeric sort. For example, if the foo series have bar label values of 1, 101, 15, and 2, sort_by_label_numeric(foo, "bar") returns the series in the order of bar label values: 101, 15, 2, and 1.

See also sort_by_label_numeric and sort_by_label_desc.

Aggregate Functions¶

Aggregate functions calculate aggregations over groups of Rollup results.

Additional details:

By default, aggregation is performed on a single group. Multiple independent groups can be specified by using the by and without modifiers. For example, count(up) by (job) groups the Rollup results by the job label value and calculates the count aggregate function separately within each group, while count(up) without (instance) groups the Rollup results by all labels except instance before calculating the count aggregate function. Multiple labels can be placed in the by and without modifiers.
If an aggregate function is applied directly to a series selector, the default_rollup() function is automatically applied before the aggregation. For example, count(up) is implicitly converted to count(default_rollup(up[1i])).
Aggregate functions accept any number of arguments. For example, avg(q1, q2, q3) returns the average of each point for q1, q2, and q3.
Aggregate functions support an optional limit N suffix, which can be used to limit the number of output groups. For example, sum(x) by (y) limit 3 limits the number of aggregated groups to 3. All other groups are ignored.

See also implicit query conversions.

Supported Aggregate Functions List¶

any¶

any(q) by (group_labels) is an aggregate function that returns one series per group_labels from the time series returned by q.

See also group.

avg¶

avg(q) by (group_labels) is an aggregate function that returns the average of the time series of q for each group_labels. The aggregation is calculated separately for each group of points with the same timestamp.

This function is supported by PromQL.

bottomk¶

bottomk(k, q) is an aggregate function that returns the k points with the smallest values from all time series of q. The aggregation is calculated separately for each group of points with the same timestamp.

This function is supported by PromQL. See also topk.

bottomk_avg¶

bottomk_avg(k, q, "other_label=other_value") is an aggregate function that returns the k time series with the smallest average values in q. If the optional other_label=other_value argument is set, the sum of the remaining time series is returned with the given label. For example, bottomk_avg(3, sum(process_resident_memory_bytes) by (job), "job=other") returns the 3 time series with the smallest average values and a time series with the label {job="other"} containing the sum of any remainder.

See also topk_avg.

bottomk_last¶

bottomk_last(k, q, "other_label=other_value") is an aggregate function that returns the k time series with the smallest last values in q. If the optional other_label=other_value argument is set, the sum of the remaining time series is returned with the given label. For example, bottomk_max(3, sum(process_resident_memory_bytes) by (job), "job=other") returns the 3 time series with the smallest maximum values and a time series with the label {job="other"} containing the sum of any remainder.

See also topk_last.

bottomk_max¶

bottomk_max(k, q, "other_label=other_value") is an aggregate function that returns the k time series with the smallest maximum values in q. If the optional other_label=other_value argument is set, the sum of the remaining time series is returned with the given label. For example, bottomk_max(3, sum(process_resident_memory_bytes) by (job), "job=other") returns the 3 time series with the smallest maximum values and a time series with the label {job="other"} containing the sum of any remainder.

See also topk_max.

bottomk_median¶

bottomk_median(k, q, "other_label=other_value") is an aggregate function that returns the k time series with the smallest median values in q. If the optional other_label=other_value argument is set, the sum of the remaining time series is returned with the given label. For example, bottomk_median(3, sum(process_resident_memory_bytes) by (job), "job=other") returns the 3 time series with the smallest median values and a time series with the label {job="other"} containing the sum of any remainder.

See also topk_median.

bottomk_min¶

bottomk_min(k, q, "other_label=other_value") is an aggregate function that returns the k time series with the smallest minimum values in q. If the optional other_label=other_value argument is set, the sum of the remaining time series is returned with the given label. For example, bottomk_min(3, sum(process_resident_memory_bytes) by (job), "job=other") returns the 3 time series with the smallest minimum values and a time series with the label {job="other"} containing the sum of any remainder.

See also topk_min.

count¶

count(q) by (group_labels) is an aggregate function that returns the number of non-null points of q for each group_labels. The aggregation is calculated separately for each group of points with the same timestamp.

This function is supported by PromQL.

count_values¶

count_values("label", q) is an aggregate function that counts the number of points with the same value and stores the count in a time series with an additional label for each initial value. The aggregation is calculated separately for each group of points with the same timestamp.

This function is supported by PromQL.

distinct¶

distinct(q) is an aggregate function that counts the number of unique values for each group of points with the same timestamp.

geomean¶

geomean(q) is an aggregate function that calculates the geometric mean for each group of points with the same timestamp.

group¶

group(q) by (group_labels) is an aggregate function that returns 1 for each group_labels from the time series returned by q.

This function is supported by PromQL. See also any.

histogram¶

histogram(q) is an aggregate function that calculates a VictoriaMetrics histogram for each group of points with the same timestamp. Visualize large numbers of time series with a heatmap. For more details, see this article.

See also histogram_over_time and histogram_quantile.

limitk¶

limitk(k, q) by (group_labels) is an aggregate function that returns up to k time series from q for each group_labels. The set of returned time series remains consistent across calls.

See also limit_offset.

mad¶

mad(q) by (group_labels) is an aggregate function that calculates the median absolute deviation for all time series returned by q for each group_labels. The aggregation is calculated separately for each group of points with the same timestamp.

See also range_mad, mad_over_time, outliers_mad, and stddev.

max¶

max(q) by (group_labels) is an aggregate function that returns the maximum value for each group_labels from all time series returned by q. The aggregation is calculated separately for each group of points with the same timestamp.

This function is supported by PromQL.

median¶

median(q) by (group_labels) is an aggregate function that returns the median value for each group_labels from all time series returned by q. The aggregation is calculated separately for each group of points with the same timestamp.

min¶

min(q) by (group_labels) is an aggregate function that returns the minimum value for each group_labels from all time series returned by q. The aggregation is calculated separately for each group of points with the same timestamp.

This function is supported by PromQL.

mode¶

mode(q) by (group_labels) is an aggregate function that returns the mode for each group_labels from all time series returned by q. The aggregation is calculated separately for each group of points with the same timestamp.

outliers_mad¶

outliers_mad(tolerance, q) is an aggregate function that returns time series from q that have at least one point outside the median absolute deviation (MAD) multiplied by tolerance. For example, it returns time series with at least one point below median(q)-mad(q) or above median(q)+mad(q).

See also outliersk and mad.

outliersk¶

outliersk(k, q) is an aggregate function that returns up to k time series with the largest standard deviation (i.e., outliers) from the time series returned by q.

See also outliers_mad.

quantile¶

quantile(phi, q) by (group_labels) is an aggregate function that calculates the phi quantile for each group_labels from all time series returned by q. phi must be in the range [0...1]. The aggregation is calculated separately for each group of points with the same timestamp.

This function is supported by PromQL. See also quantiles and histogram_quantile.

quantiles¶

quantiles("phiLabel", phi1, ..., phiN, q) is an aggregate function that calculates the phi* quantiles from all time series returned by q and returns them in time series with {phiLabel="phi*"} labels. The phi* values must be in the range [0...1]. The aggregation is calculated separately for each group of points with the same timestamp.

See also quantile.

share(q) by (group_labels) is an aggregate function that returns the share in the range [0..1] for each non-negative point at each timestamp, such that the sum of shares per group_labels is 1.

This function is used to normalize histogram bucket shares to the [0..1] range:

share(
  sum(
    rate(http_request_duration_seconds_bucket[5m])
  ) by (le, vmrange)
)

See also range_normalize.

stddev¶

stddev(q) by (group_labels) is an aggregate function that calculates the standard deviation for each group_labels from all time series returned by q. The aggregation is calculated separately for each group of points with the same timestamp.

This function is supported by PromQL.

stdvar¶

stdvar(q) by (group_labels) is an aggregate function that calculates the standard variance for each group_labels from all time series returned by q. The aggregation is calculated separately for each group of points with the same timestamp.

This function is supported by PromQL.

sum¶

sum(q) by (group_labels) is an aggregate function that returns the sum for each group_labels from all time series returned by q. The aggregation is calculated separately for each group of points with the same timestamp.

This function is supported by PromQL.

sum2¶

sum2(q) by (group_labels) is an aggregate function that calculates the sum of squares for each group_labels from all time series returned by q. The aggregation is calculated separately for each group of points with the same timestamp.

topk¶

topk(k, q) is an aggregate function that returns the top k points with the largest values from all time series returned by q. The aggregation is calculated separately for each group of points with the same timestamp.

This function is supported by PromQL. See also bottomk.

topk_avg¶

topk_avg(k, q, "other_label=other_value") is an aggregate function that returns the top k time series with the largest average values in q. If an optional other_label=other_value argument is set, the sum of the remaining time series is returned with the given label. For example, topk_avg(3, sum(process_resident_memory_bytes) by (job), "job=other") returns the three largest average time series and a time series with the label {job="other"} containing the sum of the remaining series (if any).

See also bottomk_avg.

topk_last¶

topk_last(k, q, "other_label=other_value") is an aggregate function that returns the top k time series with the largest last values in q. If an optional other_label=other_value argument is set, the sum of the remaining time series is returned with the given label. For example, topk_max(3, sum(process_resident_memory_bytes) by (job), "job=other") returns the three largest maximum time series and a time series with the label {job="other"} containing the sum of the remaining series (if any).

See also bottomk_last.

topk_max¶

topk_max(k, q, "other_label=other_value") is an aggregate function that returns the top k time series with the largest maximum values in q. If an optional other_label=other_value argument is set, the sum of the remaining time series is returned with the given label. For example, topk_max(3, sum(process_resident_memory_bytes) by (job), "job=other") returns the three largest maximum time series and a time series with the label {job="other"} containing the sum of the remaining series (if any).

See also bottomk_max.

topk_median¶

topk_median(k, q, "other_label=other_value") is an aggregate function that returns the top k time series with the largest median values in q. If an optional other_label=other_value argument is set, the sum of the remaining time series is returned with the given label. For example, topk_median(3, sum(process_resident_memory_bytes) by (job), "job=other") returns the three largest median time series and a time series with the label {job="other"} containing the sum of the remaining series (if any).

See also bottomk_median.

topk_min¶

topk_min(k, q, "other_label=other_value") is an aggregate function that returns the top k time series with the largest minimum values in q. If an optional other_label=other_value argument is set, the sum of the remaining time series is returned with the given label. For example, topk_min(3, sum(process_resident_memory_bytes) by (job), "job=other") returns the three largest minimum time series and a time series with the label {job="other"} containing the sum of the remaining series (if any).

See also bottomk_min.

zscore¶

zscore(q) by (group_labels) is an aggregate function that returns the z-score values for each group_labels from all time series returned by q. The aggregation is calculated separately for each group of points with the same timestamp. This function is useful for detecting outliers in correlated groups of time series.

See also zscore_over_time and range_trim_zscore.

Subqueries¶

MetricsQL supports and extends PromQL subqueries. For details, see this article. Any Rollup function that is not a series selector forms a subquery. Multiple layers of Rollup functions can be implicitly converted via implicit query conversions. For example, delta(sum(m)) is implicitly converted to delta(sum(default_rollup(m[1i]))[1i:1i]), so it becomes a subquery because it contains nested default_rollup and delta.

MetricsQL executes subqueries as follows:

The inner Rollup function is calculated using the step value of the outer Rollup function. For example, for the expression max_over_time(rate(http_requests_total[5m])[1h:30s]), the inner function rate(http_requests_total[5m]) is calculated with step=30s. The resulting data points are aligned by step.
The outer Rollup function is calculated on the result of the inner Rollup function using the step value passed from the frontend.

Implicit Query Conversions¶

Before starting calculation, MetricsQL performs the following implicit conversions on incoming queries:

If the lookbehind window in square brackets is missing inside a Rollup function, [1i] is automatically added. [1i] represents a step value passed to /api/v1/query_range. In Grafana, it is also known as $__interval. For example, rate(http_requests_count) is automatically converted to rate(http_requests_count[1i]).
All series selectors not wrapped in a Rollup function are automatically wrapped in the default_rollup function. For example:
- foo is converted to default_rollup(foo[1i])
- foo + bar is converted to default_rollup(foo[1i]) + default_rollup(bar[1i])
- count(up) is converted to count(default_rollup(up[1i])), because count is not a Rollup function but an aggregate function
- abs(temperature) is converted to abs(default_rollup(temperature[1i])), because abs is not a Rollup function but a transform function
If the step in square brackets is missing in a subquery, a 1i step is automatically added. For example, avg_over_time(rate(http_requests_total[5m])[1h]) is automatically converted to avg_over_time(rate(http_requests_total[5m])[1h:1i]).
If something other than a series selector is passed to a Rollup function, a subquery with a 1i lookbehind window and a 1i step is automatically formed. For example, rate(sum(up)) is automatically converted to rate((sum(default_rollup(up[1i])))[1i:1i]).