DQL Function Reference¶
DQL provides a rich set of functions for data aggregation, transformation, and matching. This document describes the semantics, parameters, and usage of each function in detail.
Aggregation Functions¶
Aggregation functions aggregate multiple rows of data into a single value, typically used with a time window (time-expr) and grouping (BY clause).
Basic Aggregation¶
sum¶
Calculates the sum of field values.
Syntax:
Parameters:
field: Numeric field
Examples:
// Calculate total request count
M::http_requests:(sum(request_count)) [1h]
// Calculate total request count per service
M::http_requests:(sum(request_count)) [1h] BY service
avg¶
Calculates the average of field values.
Syntax:
Parameters:
field: Numeric field
Examples:
// Calculate average response time
M::response_time:(avg(duration)) [1h] BY endpoint
// Calculate average CPU usage
M::cpu:(avg(usage)) [1h] BY host
count¶
Counts the number of data rows.
Syntax:
Parameters:
field: Any field, counts non-null values*: Counts all rows
Examples:
// Count log entries
L::nginx:(count(*)) [1h]
// Count requests with response time
M::response_time:(count(duration)) [1h] BY service
min / max¶
Calculates the minimum or maximum value of a field.
Syntax:
Parameters:
field: Numeric field
Examples:
// Find the maximum response time
M::response_time:(max(duration)) [1h] BY endpoint
// Find the range of CPU usage
M::cpu:(min(usage), max(usage)) [1h] BY host
first / last¶
Gets the first or last value (in time order).
Syntax:
Parameters:
field: Any field
Notes:
first: Returns the earliest valuelast: Returns the latest value; if the field is an array type, it will be expandedlast_row: Returns the latest value; array types are not expanded
Examples:
// Get the latest status value
M::system:(last(status)) [1h] BY host
// Get the initial and final values
M::counter:(first(value), last(value)) [1h] BY metric
any¶
Returns any non-null value. Useful for retrieving sample data when no specific aggregation order is needed.
Syntax:
Parameters:
field: Any field
Examples:
// Get any message sample
L::logs:(any(message)) [1h] BY service
// Get any error stack trace
L::error_logs:(any(stack_trace)) [1h] BY error_type
SHIFT¶
Expression-level SHIFT references aggregated values from a historical window in the projection, aligning them with the current aggregation result:
Syntax:
Current values, historical values, and derived calculations are all explicitly expressed in the projection expression. When referencing the same historical value multiple times, write the same SHIFT expression repeatedly:
Example:
L::logs:(
service,
count(*) AS requests,
count(*) SHIFT 7d AS requests_last_week,
CASE
WHEN (count(*) SHIFT 7d) = nil OR (count(*) SHIFT 7d) = 0 THEN nil
ELSE count(*) / (count(*) SHIFT 7d)
END AS requests_ratio
)[1d:1h] BY service
duration must be a positive fixed duration, e.g., 1h, 7d.
Usage Restrictions:
- Can only appear in a projection and only apply to an aggregation expression whose row identity remains stable across windows within the current projection, or to a subquery measure derived from such an aggregation;
- Cannot be applied to a projection alias, dimension column,
WHERE,BY,HAVING,ORDER BY,SORDER BYjust declared in the sameSELECT, nor canSHIFTbe nested; - Aggregations whose output rows are determined by each window's data, such as
distinct,distinct_by_collapse,field_values,histogram, cannot be used as operands; expressions likeuint(field)that only transform raw fields are not aggregation measures, while scalar transformations wrapping stable aggregations likeabs(sum(field))can be used; - When filtering or sorting based on historical values, do it in an outer
SELECT; - All expression-level
SHIFTin the sameSELECTcan contain at most 16 different offsets in total.
Query-level SHIFT (see Time Shift in DQL Main Document) can be combined with expression-level SHIFT: the query-level offset first determines the base window of the entire query, then the expression-level offset reads earlier aggregated values relative to that base window.
spread¶
Calculates the range (difference between maximum and minimum).
Syntax:
Parameters:
field: Numeric field
Examples:
// Calculate the fluctuation range of response time
M::response_time:(spread(duration)) [1h] BY endpoint
stddev¶
Calculates the standard deviation.
Syntax:
Parameters:
field: Numeric field
Examples:
// Calculate the standard deviation of response time
M::response_time:(stddev(duration)) [1h] BY endpoint
mode¶
Calculates the mode (the value that appears most frequently).
Syntax:
Parameters:
field: Any field
Examples:
count_series¶
Counts the number of time series (groups). Returns how many distinct time series exist within the current query range.
Syntax:
Parameters:
field: Any field (typically*or any existing field)
Examples:
// Count how many hosts are reporting CPU metrics
M::cpu:(count_series(*)) [1h]
// Count how many instances each service has
M::http_requests:(count_series(*)) [1h] BY service
Statistical Aggregation (Estimation Functions)¶
The following functions use probabilistic data structures for estimation, suitable for large data volumes, balancing accuracy and performance.
count_distinct¶
Estimates the number of distinct values of a field.
Syntax:
Parameters:
field: Any field
Algorithm Notes:
Uses the HyperLogLog algorithm for cardinality estimation: - Number of registers: 2¹⁶ = 65536 - Uses LogLog-Beta estimation method - Standard error: approximately 0.4%
Use Cases:
- Count unique users (UV)
- Count distinct IP addresses
- Analyze unique request IDs
Examples:
// Count unique users
L::access_logs:(count_distinct(user_id)) [1d] BY service
// Count distinct IP addresses
L::nginx:(count_distinct(client_ip)) [1h] BY endpoint
percentile¶
Estimates the percentile of a field.
Syntax:
Parameters:
field: Numeric fieldn: Percentile, range 0-100
Shorthand Forms:
p50(field)is equivalent topercentile(field, 50)p95(field)is equivalent topercentile(field, 95)p99(field)is equivalent topercentile(field, 99)
Algorithm Notes:
Uses a log-linear interpolation histogram for estimation: - Bucket range: 10⁻⁹ to 10¹⁸, covering most numeric scenarios - 128 buckets per order of magnitude - Linear interpolation in log space improves accuracy
Use Cases:
- Calculate P99, P95 of response time
- Analyze tail latency of performance metrics
- Evaluate Service Level Agreement (SLA) compliance
Examples:
// Calculate P99 of response time
M::response_time:(percentile(duration, 99)) [1h] BY service
// Using shorthand
M::response_time:(p99(duration)) [1h] BY service
// As a Rollup function call: first compute P95 on each time series, then aggregate by service
M::response_time:(avg(duration)) [1h::5m:percentile(95)] BY service
// Calculate multiple percentiles simultaneously
M::response_time:(p50(duration), p95(duration), p99(duration)) [1h] BY service
median¶
Calculates the median, equivalent to percentile(field, 50).
Syntax:
Examples:
Histogram Functions¶
DQL provides three histogram-related functions for different data sources and scenarios:
| Function | Use Case | Data Source Type | Recommendation |
|---|---|---|---|
histogram_auto |
Value distribution statistics for detail data (logs, Traces) | Detail model (Logs/Trace) | ⭐⭐⭐ Recommended |
histogram |
Histogram with fixed bucket boundaries | Detail model (Logs/Trace) | ⭐⭐ Deprecated |
histogram_quantile |
Calculate quantiles from Prometheus histogram metrics | Prometheus metrics | ⭐⭐⭐ Recommended |
histogram_auto (Recommended)¶
Automatically generates a distribution histogram, designed for value distribution statistics of detail data such as logs and Traces.
Features:
- No need to specify bucket boundaries; automatically adapts to data distribution
- Uses a log-linear interpolation histogram algorithm, covering the numeric range from 10⁻⁹ to 10¹⁸
- Returns both quantile statistics and bucket distribution information
Syntax:
Parameters:
field: Numeric field
Return Values:
| Column | Description |
| ---------------- | ---------------------------- |
| lower_bounds | Array of lower bucket bounds |
| upper_bounds | Array of upper bucket bounds |
| counts | Array of bucket counts |
| min | Minimum value |
| p50 | 50th percentile |
| p75 | 75th percentile |
| p90 | 90th percentile |
| p95 | 95th percentile |
| p99 | 99th percentile |
| max | Maximum value |
Algorithm Notes: Uses an estimated histogram (log-linear interpolation) with 128 buckets per order of magnitude, suitable for distribution statistics on large datasets.
Use Cases:
- Analyze response time distribution in logs
- Analyze duration distribution in Traces
- Exploratory data analysis without preset bucket boundaries
Examples:
// Analyze response time distribution in Nginx access logs
L::nginx:(histogram_auto(response_time)) [1h]
// Analyze request duration distribution per service
L::app_logs:(histogram_auto(duration)) [1h] BY service
// Analyze duration distribution of Trace calls
T::http_client:(histogram_auto(elapsed)) [1h] BY operation
Result Example:
| lower_bounds | upper_bounds | counts | min | p50 | p75 | p90 | p95 | p99 | max |
|---|---|---|---|---|---|---|---|---|---|
| [0, 10, 100] | [10, 100, 1000] | [1000, 500, 100] | 0.5 | 45 | 120 | 280 | 450 | 850 | 1200 |
Note:
lower_bounds,upper_bounds,countsare array types representing the boundaries and counts of each bucket.
histogram (Deprecated)¶
Generates a histogram with specified bucket boundaries. This function is Deprecated; use histogram_auto instead.
Notes:
histogram requires manually specifying bucket boundary parameters, which is less flexible. histogram_auto automatically adapts to data distribution, covers a wider range of values, and returns richer statistical information.
Syntax:
Parameters:
field: Numeric fieldleft_bound: Left boundaryright_bound: Right boundarybucket_size: Bucket sizethreshold(optional): Minimum count per bucket; buckets below this value are not returned
Return Values:
Returns two columns: bucket_le (upper boundary of the bucket) and count (count)
Examples:
// Generate a histogram with range 0-1000ms and 100ms per bucket
M::response_time:(histogram(duration, 0, 1000, 100)) [1h]
// Recommended replacement using histogram_auto
M::response_time:(histogram_auto(duration)) [1h]
Result Example:
| bucket_le | count |
|---|---|
| 100 | 1500 |
| 200 | 2800 |
| 300 | 3500 |
| ... | ... |
| 1000 | 5000 |
histogram_quantile¶
Calculates quantiles from Prometheus histogram metrics.
Features:
- Specifically designed for processing histogram-type metrics reported by Prometheus
- Relies on the
lelabel (orvmrangelabel for VictoriaMetrics) to identify bucket boundaries - Input data should be cumulative counts
Syntax:
Parameters:
field: Histogram count field (e.g.,http_request_duration_bucket)q: Quantile, range 0-1 (e.g., 0.99 for P99)
Use Case Comparison:
| Scenario | Recommended Function | Notes |
|---|---|---|
| Analyze response time distribution in logs | histogram_auto |
Logs are detail data, no pre-aggregated histogram |
| Calculate P99 of Prometheus histogram metrics | histogram_quantile |
Metrics are pre-aggregated with le label |
| Analyze duration distribution of Trace calls | histogram_auto |
Traces are detail data |
le Label Handling Mechanism:
histogram_quantile relies on the le label (less than or equal) to identify histogram bucket boundaries:
- Prometheus format (default): Uses the
lelabel directly to represent the bucket upper boundary levalue is numeric (e.g., "0.1", "1", "10") or "+Inf" (infinity)-
Data should be cumulative counts
-
VictoriaMetrics format: Uses the
vmrangelabel to represent a range - Format:
"lower_bound...upper_bound"(e.g.,"0.1...0.2") - Data is range counts (non-cumulative)
- The function automatically converts range counts to cumulative counts
Calculation Process:
1. Sort all buckets by le value
2. If vmrange format, accumulate counts to convert to cumulative distribution
3. Ensure bucket counts are monotonically increasing (fix potential anomalous data)
4. Use linear interpolation to calculate the target quantile
Differences from PromQL:
| Feature | DQL | PromQL |
|---|---|---|
| Function Type | Aggregation function | Transformation function |
| Input Data | Directly reads metrics with le label |
Requires sum(rate(...)) by (le) |
| Usage | histogram_quantile(field, 0.99) |
histogram_quantile(0.99, sum(rate(...)) by (le)) |
| Data Format | Supports both le and vmrange labels |
Only supports le label |
| Grouping | Via DQL's BY clause | Explicit grouping via by (le) |
Equivalent Examples:
Assume a histogram metric http_request_duration_bucket with le labels (e.g., 0.1, 0.5, 1, 5, +Inf) and service label.
Scenario 1: Calculate P99 latency
DQL:
PromQL equivalent:
Scenario 2: Calculate P95 latency per service (multiple groups)
DQL:
PromQL equivalent:
Scenario 3: Calculate P50 (median) and P99
DQL:
M::http_request_duration:(
histogram_quantile(duration_bucket, 0.50) as p50,
histogram_quantile(duration_bucket, 0.99) as p99
) [1h] BY service
PromQL equivalent:
label_join(
histogram_quantile(0.50, sum(rate(http_request_duration_bucket[1h])) by (le, service)), "quantile", "", "0.50"
)
or
label_join(
histogram_quantile(0.99, sum(rate(http_request_duration_bucket[1h])) by (le, service)), "quantile", "", "0.99"
)
Note: PromQL requires
label_joinorlabel_replaceto distinguish results for different quantiles.
Notes:
- Input data must contain the
leorvmrangelabel; otherwise, calculation is impossible - If there is no
+Infbucket, the upper boundary of the last bucket is used as the maximum - Buckets with a count of 0 or NaN are skipped
- When the quantile is < 0, returns -Inf; when > 1, returns +Inf
TopN Functions¶
top¶
Gets the top N largest values.
Syntax:
Parameters:
field: Numeric fieldn: Number of values to return
Examples:
// Get the 5 requests with the longest response time
M::response_time:(top(duration, 5)) [1h] BY service
// Get the 10 hosts with the highest traffic
M::network:(top(bytes, 10)) [1h]
Result Example:
| service | top(duration, 5) |
|---|---|
| api | 1250 |
| api | 1180 |
| api | 1050 |
| api | 980 |
| api | 920 |
Note: Returns multiple rows, each containing one TopN value.
bottom¶
Gets the bottom N smallest values.
Syntax:
Parameters:
field: Numeric fieldn: Number of values to return
Examples:
// Get the 5 requests with the shortest response time
M::response_time:(bottom(duration, 5)) [1h] BY service
Result Example:
| service | bottom(duration, 5) |
|---|---|
| api | 12 |
| api | 18 |
| api | 25 |
| api | 32 |
| api | 45 |
Note: Returns multiple rows, each containing one BottomN value.
Value Collection Functions¶
distinct¶
Returns all distinct values of a field.
Syntax:
Examples:
Result Example:
| endpoint | distinct(status) |
|---|---|
| /api/v1 | 200 |
| /api/v1 | 404 |
| /api/v1 | 500 |
| /health | 200 |
Note: Returns multiple rows, each containing one distinct value.
distinct_by_collapse¶
Gets distinct values of a field using a collapse strategy, retaining the last value of other fields during deduplication.
Syntax:
Parameters:
field: Field to deduplicate bylast_fields(optional): List of fields whose last values should be retained
Notes:
Unlike distinct, distinct_by_collapse retains information from other related fields (taking the last value) during deduplication, suitable for scenarios where context information needs to be preserved.
Examples:
// Get distinct user IDs, retaining the last access time for each user
L::access_logs:(distinct_by_collapse(user_id, [timestamp])) [1h]
// Get distinct hosts, retaining the last status and message
O::HOST:(distinct_by_collapse(host, [status, message])) [1h]
Result Example:
| user_id | last(timestamp) | last(path) |
|---|---|---|
| user001 | 1704067200000 | /checkout |
| user002 | 1704067100000 | /product |
| user003 | 1704067000000 | /home |
Note: Returns the deduplicated primary field value, along with the last values of the fields specified in
last_fields.
collect¶
Collects all values (including duplicates).
Syntax:
Parameters:
field: Any fieldlimit(optional): Maximum number of values to collect
Examples:
// Collect all response times
M::response_time:(collect(duration)) [1h] BY service
// Collect at most 100 values
M::response_time:(collect(duration, 100)) [1h] BY service
Result Example:
| service | collect(duration) |
|---|---|
| api | [120, 135, 98, 142, ...] |
| web | [45, 52, 48, 61, ...] |
Note: Returns an array type containing all collected values (may include duplicates).
collect_distinct¶
Collects all distinct values.
Syntax:
Parameters:
field: Any fieldlimit(optional): Maximum number of values to collect
Examples:
Result Example:
| service | collect_distinct(error_type) |
|---|---|
| api | ["timeout", "connection refused", "404"] |
| web | ["200", "301", "404"] |
Note: Returns an array type containing all deduplicated values.
field_values¶
Gets all values of a field, returning an array type.
Syntax:
Examples:
Result Example:
| metric_name | field_values(tags) |
|---|---|
| cpu_usage | ["host:A", "env:prod", "team:backend"] |
| memory_used | ["host:B", "env:staging", "team:frontend"] |
Note: Returns an array type containing all values of the field.
Filtered Aggregation¶
count_filter¶
Counts the number of field values that are in a specified list.
Syntax:
Parameters:
field: Any fieldvalues: List of values
Examples:
// Count requests with specific status codes
M::http:(count_filter(status, [200, 201, 204])) [1h] BY endpoint
// Count error-level logs
L::logs:(count_filter(level, ["error", "critical"])) [1h] BY service
Helper Functions¶
default¶
Sets a default value for a field, returning the default when the field is null.
Syntax:
Parameters:
field: Any fielddefault_value: Default value (can be numeric, string, boolean, or null)
Examples:
// Set a default response time for null values
M::response_time:(default(duration, 0)) [1h] BY service
Time Series Functions¶
Time series functions are used to process data that changes over time, especially Counter-type metrics.
Rollup Functions¶
Rollup functions are used to preprocess raw time series data within a time window. For detailed information, please refer to Rollup Functions in this document.
Syntax Notes:
- Rollup is written in the time clause, e.g.,
[rate],[1h::5m:rate]. - Written outside the query (e.g.,
rate(DQL)) is an outer function, not a Rollup. - For differences in execution phase and parameters between Rollup shorthand, Rollup function calls, and explicit aggregation calls, please refer to the DQL Main Document's Rollup Functions.
The time clause supports Rollup shorthand and also allows passing additional algorithm parameters to Rollup functions, e.g.,
[1h::1m:ewma(0.3)]. Parameters in the time clause only represent algorithm parameters; the input field is still determined by the Select field. Functions that require multiple input fields should use explicit aggregation calls in the Select clause.
Functions Supporting Rollup Shorthand:
| Function | Description |
|---|---|
rate |
Calculates rate of change (per second) |
irate |
Calculates instant rate of change |
increase |
Calculates increase |
deriv |
Calculates derivative (rate of change) |
difference |
Calculates difference |
non_negative_derivative |
Calculates non-negative derivative |
non_negative_difference |
Calculates non-negative difference |
rate_over_sum |
Calculates per-second average |
rate_over_count |
Calculates per-second count |
sum |
Sum |
avg |
Average |
min |
Minimum |
max |
Maximum |
count |
Count |
first |
First value |
last |
Last value |
stddev |
Standard deviation |
mode |
Mode |
spread |
Range |
any |
Any value |
slope |
Linear trend slope |
zscore |
Latest point Z-Score |
mad_score |
Latest point MAD anomaly score |
change_score |
Series change score |
Functions Supporting Rollup Function Calls:
| Function | Description |
|---|---|
ewma(alpha) |
Exponentially Weighted Moving Average |
moving_average(n) |
Moving average |
percentile(p) |
Percentile |
Examples:
// Calculate request QPS
M::http_requests:(sum(request_count)) [1h::5m:rate] BY service
// Shorthand
M::cpu:(max(usage)) [rate]
// Rollup function call with algorithm parameter
M::cpu:(avg(usage)) [1h::1m:ewma(0.3)] BY host
Rate Calculation¶
rate¶
Calculates the rate of change of a metric (per second).
Syntax:
Notes:
rate calculates the average rate of change of a Counter metric within a time window. For monotonically increasing Counter-type metrics, aggregating raw values directly is meaningless; the rate of change must be calculated first.
Use Cases:
- Calculate request QPS
- Calculate data ingestion rate
- Analyze traffic growth trends
Examples:
// Calculate request QPS
M::http_requests:(sum(request_count)) [rate] BY service
// Calculate data ingestion rate
M::data_ingestion:(sum(bytes)) [rate] BY source
irate¶
Calculates the instant rate of change of a metric.
Syntax:
Notes:
Unlike rate, irate only uses the last two data points to calculate the rate of change, reflecting the instantaneous rate of change, making it more suitable for alerting scenarios.
Examples:
increase¶
Calculates the increase of a metric.
Syntax:
Notes:
increase returns the total increase within the time window, not the rate of change.
Examples:
// Calculate total request count increase
M::http_requests:(sum(request_count)) [increase] BY service
rate_over_sum¶
Calculates the per-second average value (sum / time window seconds).
Syntax:
Notes:
Equivalent to sum(field) / time_window_seconds, used to calculate the per-second average. Often used in the Rollup phase to convert cumulative values to a per-second rate.
Difference from rate:
rate: Calculates the rate of change of a Counter (handles resets)rate_over_sum: Simply divides the sum by the time window seconds
Examples:
// Calculate average requests per second
M::http_requests:(rate_over_sum(request_count)) [1h] BY service
rate_over_count¶
Calculates the per-second count (count / time window seconds).
Syntax:
Notes:
Equivalent to count(field) / time_window_seconds, used to calculate the number of occurrences per second.
Examples:
Difference Calculation¶
This section describes function semantics. The same function can be used both as a Rollup (e.g., [rate], [increase]) and in a query expression (e.g., rate(field), increase(field)); the execution phase differs, so choose the position based on business requirements.
rate / deriv¶
Calculates the rate of change (derivative). rate is used for Counter-type metrics (ignores negative values), while deriv is used for Gauge-type metrics (preserves negative values).
Aliases:
rateis aliased asnon_negative_derivative;derivis aliased asderivative(PromQL style)
Syntax:
// Counter metric: calculate non-negative rate of change (ignores negative values due to resets)
rate(field)
// Gauge metric: calculate full rate of change (includes negative values)
deriv(field)
Function Selection:
| Function | Description | Use Case |
|---|---|---|
rate |
Calculates only non-negative rate of change | Counter-type metrics (monotonically increasing) |
deriv |
Calculates full rate of change (including negative values) | Gauge-type metrics (can increase or decrease) |
Examples:
// Counter metric: calculate request QPS
M::requests:(rate(count)) [1h::5m] BY service
// Gauge metric: calculate memory usage rate of change
M::memory:(deriv(used)) [1h::5m] BY host
increase / difference¶
Calculates the difference between consecutive values. increase is used for Counter-type metrics (ignores negative values), while difference is used for Gauge-type metrics (preserves negative values).
Note:
increaseanddifferenceare two independent functions with different behaviors; they are not aliases.
Syntax:
// Counter metric: calculate non-negative difference (ignores negative values due to resets)
increase(field)
// Gauge metric: calculate full difference (includes negative values)
difference(field)
Function Selection:
| Function | Description | Use Case |
|---|---|---|
increase |
Calculates only non-negative difference | Counter-type metrics (monotonically increasing) |
difference |
Calculates full difference (including negative values) | Gauge-type metrics (can increase or decrease) |
Examples:
// Counter metric: calculate request increase
M::requests:(increase(count)) [1h::5m] BY service
// Gauge metric: calculate request count change (may increase or decrease)
M::requests:(difference(count)) [1h::5m] BY service
Moving Calculations¶
moving_average¶
Calculates the moving average.
Syntax:
Parameters:
field: Numeric fieldn: Window size (number of data points)
Examples:
// Calculate 5-point moving average
M::cpu:(moving_average(usage, 5)) [1h::1m] BY host
// As a Rollup function call: first compute 5-point moving average on each time series, then aggregate by host
M::cpu:(avg(usage)) [1h::1m:moving_average(5)] BY host
Time Series Analysis Aggregation¶
The following functions are used as aggregation functions within a query, computing a single numeric result from the value sequence in each time window and group.
ewma¶
Calculates the Exponentially Weighted Moving Average (EWMA). alpha is the smoothing coefficient and must be explicitly provided.
Syntax:
Parameters:
field: Numeric fieldalpha: Smoothing coefficient, range(0, 1]; larger values give higher weight to recent data points
Notes:
ewmahas no implicit defaultalpha;ewma(field)will result in an error.- When used as a Rollup in the time clause, write it as
[...:ewma(alpha)]; parameters in the time clause only passalpha, not the field name. ewmarequiresalpha, so it does not support the Rollup shorthand[...:ewma]without parameters.
Examples:
// Explicit aggregation call: compute EWMA in the Select aggregation phase
M::cpu:(ewma(usage, 0.3)) [1h::1m] BY host
// Rollup function call: first compute EWMA on each time series, then aggregate by host
M::cpu:(avg(usage)) [1h::1m:ewma(0.3)] BY host
slope¶
Calculates the linear trend slope of the sequence over time, with time in seconds.
Syntax:
Notes:
- At least 2 valid points are required.
- Returns null when there is no time change or insufficient valid points.
Examples:
zscore¶
Calculates the Z-Score of the latest point relative to the mean and standard deviation within the window.
Syntax:
Notes:
- Result:
(latest - mean) / stddev. - At least 2 valid points are required; returns null when the standard deviation is 0.
Examples:
// Evaluate the deviation of the latest response time relative to the historical window
M::response_time:(zscore(duration)) [1h::5m] BY service
mad_score¶
Calculates the MAD (Median Absolute Deviation) anomaly score of the latest point.
Syntax:
Notes:
- Uses the median and MAD to measure the deviation of the latest point, which is more robust to outliers than mean/standard deviation.
- At least 2 valid points are required; returns null when the MAD is 0.
Examples:
// Detect an anomaly in the latest latency point using a robust anomaly score
M::latency:(mad_score(p95)) [1h::5m] BY service
change_score¶
Calculates the change score of a sequence within a window, used to detect time series where the mean has significantly shifted.
Syntax:
Notes:
- Enumerates possible split points, compares the mean difference between the left and right segments, and normalizes by the pooled standard deviation.
- At least 4 valid points are required; returns null when insufficient.
Examples:
// Detect whether the error rate has changed abruptly within the window
M::error_rate:(change_score(value)) [1h::5m] BY service
corr¶
Calculates the Pearson correlation coefficient between two numeric fields.
Syntax:
Notes:
- Return value range is typically
[-1, 1]. - At least 2 pairs of valid points are required; returns null when either field has no variation.
corrrequires two input fields and does not support the time clause Rollup syntax.
Examples:
// Calculate the correlation between CPU usage and request count
M::service_metric:(corr(cpu_usage, request_count)) [1h::5m] BY service
cumsum¶
Calculates the cumulative sum.
Syntax:
Examples:
Transformation Functions¶
Transformation functions are used to perform mathematical operations, type conversions, or string processing on field values.
Mathematical Functions¶
abs¶
Calculates the absolute value.
Syntax:
Examples:
round / ceil / floor¶
Rounding functions.
Syntax:
round(field) // Round to nearest integer
round(field, digits) // Round to specified decimal places
ceil(field) // Round up
floor(field) // Round down
Examples:
// Round up response time
M::response_time:(ceil(duration)) [1h]
// Round percentage
M::cpu:(round(usage)) [1h] BY host
// Round average response time to 2 decimal places
L::log:(round(avg(duration), 2)) BY api
log / log2 / log10¶
Logarithmic functions.
Syntax:
Examples:
Type Conversion¶
int / uint / float / string / bool¶
Type conversion functions.
Syntax:
int(field) // Convert to signed integer
uint(field) // Convert to unsigned integer
float(field) // Convert to floating point
string(field) // Convert to string
bool(field) // Convert to boolean
Examples:
// Convert string to numeric
L::logs:(int(response_time)) [1h] BY service
// Convert numeric to string for concatenation
M::metrics:(string(value)) [1h] BY metric_name
String Functions¶
lower¶
Converts a string to lowercase.
Syntax:
Parameters:
field: String field
Return Value: Returns the converted lowercase string.
upper¶
Converts a string to uppercase.
Syntax:
Parameters:
field: String field
Return Value: Returns the converted uppercase string.
trim¶
Removes leading and trailing whitespace from a string.
Syntax:
Parameters:
field: String field
Return Value: Returns the string with leading and trailing whitespace removed.
ltrim¶
Removes leading whitespace from a string.
Syntax:
Parameters:
field: String field
Return Value: Returns the string with leading whitespace removed.
rtrim¶
Removes trailing whitespace from a string.
Syntax:
Parameters:
field: String field
Return Value: Returns the string with trailing whitespace removed.
length¶
Returns the length of a string (in characters).
Syntax:
Parameters:
field: String field
Return Value: Returns the string length.
substr¶
Extracts a substring.
Syntax:
Parameters:
field: String fieldstart: Starting position (0-based, negative values indicate position from the end)length(optional): Substring length
Return Value: Returns the extracted substring.
Examples:
// Extract the first 100 characters of a message
L::logs:(substr(message, 0, 100)) [1h]
// Extract the last 10 characters
L::logs:(substr(message, -10)) [1h]
Result Example:
| message | substr(message, 0, 10) | substr(message, -5) |
|---|---|---|
| "Error: connection timeout" | "Error: con" | "eout" |
regexp_extract¶
Regular expression extraction.
Syntax:
Parameters:
field: String fieldpattern: Regular expressionn(optional): Extract the nth capturing group, default is 0 (the entire match)
Return Value: Returns a single string containing the content of the nth capturing group. Returns null if no match.
Examples:
// Extract error code
L::logs:(regexp_extract(message, 'error_code: (\d+)', 1)) [1h]
// Extract IP address
L::nginx:(regexp_extract(message, '(\d+\.\d+\.\d+\.\d+)', 1)) [1h]
Result Example:
| service | regexp_extract(message, 'error_code: (\d+)', 1) |
|---|---|
| api | "404" |
| api | "500" |
| web | null |
regexp_extract_all¶
Extracts all matching results.
Syntax:
Return Value: Returns a string array containing all matching substrings.
Examples:
// Extract all numbers
L::logs:(regexp_extract_all(message, '\d+', 0)) [1h]
// Extract all IP addresses
L::logs:(regexp_extract_all(message, '\d+\.\d+\.\d+\.\d+', 0)) [1h]
Result Example:
| message | regexp_extract_all(message, '\d+.\d+.\d+.\d+', 0) |
|---|---|
| Request from 192.168.1.1 to 10.0.0.1 | ["192.168.1.1", "10.0.0.1"] |
regexp_replace¶
Replaces matched text using a regular expression.
Syntax:
Parameters:
field: String fieldpattern: Regular expressionreplacement: Replacement string; use$1,${1},$2, etc. to reference capturing groups; use${1}when the capturing group is immediately followed by a letter, digit, or underscore for disambiguation (e.g.,${1}_suffix); use$$to output a literal$
Return Value: Returns the replaced string. If no match, returns the original string.
Examples:
// Remove numbers from a message, then aggregate by prefix
L::logs:(substr(regexp_replace(message, '\\d+', ''), 0, 20) AS prefix, count(*)) [1h] BY prefix
// Normalize user IDs using capturing groups
L::logs:(regexp_replace(message, 'user=([0-9]+)', 'uid=$1') AS normalized) [1h]
md5¶
Calculates the MD5 hash.
Syntax:
Return Value: Returns a 32-character hexadecimal string (lowercase).
Examples:
Result Example:
| service | md5(message) |
|---|---|
| api | 5d41402abc4b2a76b9719d911017c592 |
| web | 098f6bcd4621d373cade4e832627b4f6 |
concat¶
String concatenation.
Syntax:
Return Value: Returns a single concatenated string.
Examples:
// Concatenate multiple fields
L::logs:(concat(service, ":", level)) [1h]
// Result: "api:error", "web:info", etc.
Result Example:
| service | level | concat(service, ":", level) |
|---|---|---|
| api | error | "api:error" |
| web | info | "web:info" |
set¶
Deduplicates and sorts an array field.
Syntax:
Return Value: Returns a deduplicated and sorted array.
Examples:
// Get all distinct tags
set(M::metrics:(tags) [1h] BY metric_name)
// Deduplicate the result of collect
set(M::http:(collect(status)) [1h] BY endpoint)
Result Example:
| metric_name | set(tags) |
|---|---|
| cpu_usage | ["env:prod", "host:A", "team:backend"] |
| memory_used | ["env:staging", "host:B", "team:frontend"] |
Log Clustering¶
drain¶
Uses the Drain algorithm to group similar logs into the same cluster, returning a representative log sample for that cluster.
Syntax:
Parameters:
field: String field or string expression, typicallymessagesimilarity_threshold: Similarity threshold, range(0, 1]; larger values mean only more similar logs are grouped togethermax_clusters: Optional, maximum number of clusters, range[1, 10000]; default is1000when omitted
Return Value:
Returns a string representative log sample. The sample is the original log at the time of cluster creation, not the generalized template maintained internally by Drain.
Algorithm Notes:
Drain is a log clustering algorithm based on a parse tree. The function continuously trains the clusterer during execution: when a new log matches an existing cluster, it returns that cluster's representative sample; when no match is found, it creates a new cluster and uses the current log as its representative sample.
Use Cases:
- Group statistics by similar logs
- Anomaly log clustering analysis
- Log noise reduction
Examples:
// Group and count by similar logs, similarity 0.7, max 1000 clusters
L::logs:(count(*)) [1h] BY drain(message, 0.7, 1000) AS sample
// Omit max_clusters, default max 1000 clusters
L::logs:(count(*)) [1h] BY drain(message, 0.9) AS sample
// Concatenate multiple fields before clustering
L::logs:(count(*)) [1h] BY drain(concat(service, " ", message), 0.7) AS sample
Result Example:
| sample | count(*) |
|---|---|
| "Request from 192.168.1.1 to /api/users took 35 ms" | 128 |
| "Query SELECT * FROM orders executed in 18 ms" | 42 |
Matching Functions¶
Matching functions are used for text matching in WHERE clauses and can also be used as expressions that return Boolean values.
Substring Matching¶
match¶
Checks if a field contains a specified substring.
Syntax:
Parameters:
pattern: Substring to matchfield: Field name (optional; can be omitted in WHERE clause)
Examples:
// Use in WHERE
L::logs:(message) {match(message, "error")} [1h]
// Shorthand
L::logs:(message) {match("error")} [1h]
// As an expression
L::logs:(match(message, "timeout")) [1h] BY match_result
Phrase Matching¶
search¶
Tokenized phrase matching, supporting mixed Chinese and English.
Syntax:
Parameters:
query: Query phrasefield: Field name (optional)
Matching Rules:
- Chinese: Tokens are segmented by character
- English: Matches by word boundaries (separated by spaces, punctuation)
- Case-insensitive
- When a non-empty
queryconsists of exactly one pair of double quotes enclosing the entire value, it is treated as a phrase match: all tokens must be adjacent and in the same order; the double quotes are not part of the match content - In the above special double-quote form, if the content is only whitespace or punctuation, it still matches as a continuous literal value
- An empty query
""is not considered a special form; it is treated as a normal tokenized search parameter - Other forms containing double quotes are valid regular tokenized search parameters and are not interpreted specially, e.g., only one double quote, extra double quotes inside the value, or double quotes not at both ends
Examples:
// Match logs containing "connection timeout"
L::logs:(message) {search("connection timeout")} [1h]
// Chinese matching
L::logs:(message) {search("连接超时")} [1h]
// Mixed Chinese and English
L::logs:(message) {search("error 错误")} [1h]
// Adjacent and ordered phrase matching; will not match "build succeeded, parse error"
L::logs:(message) {search("\"build error\"")} [1h]
// Only punctuation, matches as continuous literal substring
L::logs:(message) {search("\"---\"")} [1h]
Regular Expression Matching¶
re / regex / regexp¶
Regular expression matching, case-sensitive.
Syntax:
re(pattern)
re(field, pattern)
regex(pattern)
regex(field, pattern)
regexp(pattern)
regexp(field, pattern)
Parameters:
pattern: Regular expression (supports PromRegex syntax)field: Field name (optional)
Examples:
// Match logs starting with "error", will not match logs starting with "ERROR"
L::logs:(message) {re("error.*")} [1h]
// Match a specific error code format
L::logs:(message) {regexp(message, "ERR-\d{4}")} [1h]
// Use regex in data source
M::re('cpu.*'):(usage) [1h]
Wildcard Matching¶
wildcard¶
Wildcard pattern matching, case-sensitive.
Syntax:
Parameters:
pattern: Wildcard pattern*: Matches any number of characters?: Matches a single characterfield: Field name (optional)
Examples:
// Match messages starting with "error", will not match messages starting with "ERROR"
L::logs:(message) {wildcard("error*")} [1h]
// Match a specific format
L::logs:(message) {wildcard(message, "ERR-????")} [1h]
CIDR Matching¶
cidr¶
IP address CIDR matching.
Syntax:
Parameters:
cidr: CIDR notation network, e.g.,192.168.1.0/24field: IP address field (optional)
Examples:
// Match internal IP addresses
L::nginx:(*) {cidr(client_ip, "10.0.0.0/8")} [1h]
// Match a specific network segment
L::nginx:(*) {cidr(client_ip, "192.168.1.0/24")} [1h]
Field Existence Check¶
exists¶
Checks if a field exists. exists() is a special placeholder typically used on the right side of a comparison expression.
Syntax:
Recommended Usage:
Examples:
// Find logs with an error_type field
L::logs:(message) {error_type = exists()} [1h]
// Find logs without an error_type field
L::logs:(message) {error_type != exists()} [1h]
Query String Syntax¶
query_string¶
Uses query string syntax for complex matching.
Syntax:
Parameters:
query: Query stringfield: Field name (optional; defaults to full text search)
Query String Syntax:
1. Term Matching¶
foo # Match content containing foo
"foo bar" # Exact phrase match, must appear consecutively
foo\ bar # Escape space, match "foo bar" as a whole
2. Wildcards¶
foo* # Match content starting with foo
foo?bar # ? matches a single character
"foo*bar" # Wildcards inside quotes are not parsed, matched as literals
3. Regular Expressions¶
4. Boolean Operators¶
foo AND bar # Logical AND, both must be present
foo OR bar # Logical OR, at least one must be present
NOT foo # Logical NOT, must not contain foo
# Shorthand
foo && bar # Equivalent to foo AND bar
foo || bar # Equivalent to foo OR bar
!foo # Equivalent to NOT foo
5. Grouping¶
(foo OR bar) AND baz # Use parentheses to change precedence
!(status 429 reading) # Negate the entire expression
6. Default Operator¶
When multiple terms are separated by spaces, the default operator is OR (configurable to AND):
7. Tokenization Rules¶
The behavior of the queryString function depends on the underlying storage engine used by the current workspace:
-
ScopeDB environment:
queryStringperforms a case-insensitive contains search, i.e., it checks whether the field value contains any substring of the query string, without tokenization; -
Doris environment:
queryStringmatches based on full-text index tokenization, and its semantics are directly related to the tokenization result. The tokenizer used by Doris follows the default word boundary specification of Unicode Standard Annex #29.
Examples¶
// Simple term matching
L::logs:(message) {query_string("error timeout")} [1h]
// Boolean combination
L::logs:(message) {query_string("error AND NOT timeout")} [1h]
// Regular expression
L::logs:(message) {query_string("/ERR-\d{4}/")} [1h]
// Complex query
L::logs:(message) {query_string("(error OR warn) AND service")} [1h]
// Specify field
L::logs:(*) {query_string(message, "error AND timeout")} [1h]
// Chinese query
L::logs:(message) {query_string("错误 AND 超时")} [1h]
Outer Functions¶
Usage Recommendation: Outer functions are a legacy design. For scenarios that can be solved with Rollup + aggregation functions (e.g.,
[rate],[last],[increase], etc.), prefer the Rollup approach. Outer functions should only be used in scenarios where Rollup is not applicable (e.g., when secondary calculations on aggregation results are required). Functions likedbscan,forecastthat require center detection/prediction over the entire query result have no equivalent Rollup syntax and should be used as outer functions.
Outer functions operate on the entire DQL query result, performing secondary calculations on the output time series data. Outer functions wrap the entire DQL expression, rather than being written inside the Select clause.
Query-Internal Functions vs. Outer Functions¶
- Query-Internal Functions: Used inside a DQL expression, e.g.,
sum,avg,max, etc. - Outer Functions: Wrap the entire DQL query result, performing post-processing on the output time series.
Syntax Comparison:
// Rollup (time clause): first compute rate of change on each time series, then aggregate
M::http_requests:(sum(request_count)) [rate] BY service
// Outer function: first get the query result, then perform secondary calculation
rate(M::http_requests:(sum(request_count)) [1h::1m] BY service)
Syntax:
Examples:
// Query-internal function: average the raw data
M::cpu:(avg(usage)) [1h::5m] BY host
// Outer function: calculate moving average on the query result
moving_average(M::cpu:(avg(usage)) [1h::5m] BY host, 5)
Cumulative Calculations¶
cumsum¶
Calculates the cumulative sum, summing all previous points for each point in the time series.
Syntax:
Examples:
Difference and Derivative¶
Recommended Use: These functions should preferably be used as Rollup functions (e.g.,
[rate],[deriv]); the outer function form is only for secondary calculations on aggregation results.
The following functions can be used as outer functions:
| Function | Description |
|---|---|
derivative(DQL) |
Calculates derivative (rate of change) |
difference(DQL) |
Calculates difference from previous value |
non_negative_derivative(DQL) |
Calculates non-negative derivative |
non_negative_difference(DQL) |
Calculates non-negative difference |
rate(DQL) |
Calculates rate of change (per second) |
irate(DQL) |
Calculates instant rate of change |
Examples:
// Outer function: calculate derivative on the query result
derivative(M::cpu:(avg(usage)) [1h::5m] BY host)
// Recommended: use Rollup approach
M::cpu:(deriv(usage)) [1h::5m:last] BY host
Moving Calculations¶
moving_average¶
Calculates the moving average on the query result.
Recommended Use: Prefer the Rollup approach
moving_average(field, n); the outer function form is only for secondary smoothing of aggregation results.
Syntax:
Parameters:
DQL_expression: DQL query expressionn: Window size (number of data points)
Examples:
// Outer function: calculate moving average on the query result
moving_average(M::cpu:(avg(usage)) [1h::1m] BY host, 5)
// Recommended: use Rollup approach
M::cpu:(moving_average(usage, 5)) [1h::1m] BY host
Time Series Analysis and Anomaly Detection¶
dbscan¶
Performs DBSCAN outlier detection on the numeric time series in the query result, outputting numeric anomaly flags.
Syntax:
Parameters:
DQL_expression: DQL query expression. Typically used for multi-series query results with time windows andBYgrouping.eps: Neighborhood distance threshold, optional, default0.5, range(0, 3.0].
Return Value:
- Outputs a corresponding
dbscan(column)numeric column for each numeric column. 1indicates an outlier point,0indicates a non-outlier point.- Returns null when there are fewer than 5 valid numeric points, indicating insufficient samples for detection.
Notes:
- At least 5 valid numeric points are required; returns null when insufficient.
- The current implementation performs one-dimensional DBSCAN on each numeric column in the input table separately, preserving the time column.
Examples:
// Detect CPU usage outliers with default eps=0.5
dbscan(M::cpu:(avg(usage)) [1h::5m] BY host)
// Explicitly specify eps
dbscan(M::cpu:(avg(usage)) [1h::5m] BY host, 0.8)
forecast¶
Performs linear trend forecasting on the numeric time series in the query result, outputting predicted values for future time points.
Syntax:
Parameters:
DQL_expression: DQL query expressionsteps: Number of forecast steps, optional, default5, must be a positive integer
Return Value:
- Outputs future
stepstime points. - Each numeric column outputs a corresponding
forecast(column)numeric column. - Non-numeric columns do not participate in forecasting.
Notes:
- The current implementation uses linear trend forecasting.
- Returns null when there are fewer than 2 valid numeric points.
Examples:
// Forecast the next 5 time points
forecast(M::cpu:(avg(usage)) [1h::5m] BY host)
// Forecast the next 3 time points
forecast(M::cpu:(avg(usage)) [1h::5m] BY host, 3)
TopN¶
top / bottom¶
Recommended Use: Prefer the Rollup approach
top(field, n)orbottom(field, n); the outer function form is only for secondary filtering of aggregation results.
Gets the TopN or BottomN of the query result.
Syntax:
Examples:
// Outer function: get TopN on the query result
top(M::response_time:(max(duration)) [1h::5m] BY service, 5)
// Recommended: use Rollup approach
M::response_time:(top(duration, 5)) [1h::5m] BY service
Null Value Filling¶
fill¶
Fills null values in the query result.
For detailed information, please refer to fill function.
Examples:
// Fill null values with 0
fill(M::cpu:(avg(usage)) [1h::5m] BY host, 0)
// Linear interpolation fill
fill(M::cpu:(avg(usage)) [1h::5m] BY host, LINEAR)
Other Outer Functions¶
The following functions can also be used as outer functions:
| Function | Description |
|---|---|
abs(DQL) |
Absolute value |
round(DQL[, digits]) |
Round |
ceil(DQL) |
Round up |
floor(DQL) |
Round down |
log(DQL) / log2(DQL) / log10(DQL) |
Logarithmic transformation |
set(DQL) |
Deduplicate and sort |
concat(DQL, ...) |
String concatenation |
Combined Usage¶
Outer functions can be combined:
// Calculate moving average and round
round(moving_average(M::cpu:(avg(usage)) [1h::1m] BY host, 5))
// Calculate moving average of the rate
moving_average(rate(M::requests:(sum(count)) [1h::5m] BY service), 3)
eval Expression Calculation¶
eval is a special function that allows expression calculation outside the query, referencing results from multiple sub-queries for combined operations.
Syntax¶
Parameters:
expression: Mathematical expression, usingname.fieldto reference sub-query resultsname=(query): Named sub-queryalias: Result alias (optional)
Note:
evalalso supports the legacy syntaxname="query", but thename=(query)syntax is recommended for better type expression and readability.
How It Works¶
- Executes all named sub-queries
- Aligns the results of each sub-query by time
- Evaluates the expression for each time point
- Returns the calculation result
Use Cases¶
- Calculate ratios of multiple metrics (e.g., error rate, utilization)
- Compare metrics across different time periods
- Combine calculation results from multiple data sources
Examples¶
Calculate Error Rate¶
// Calculate error rate = error_count / total_requests * 100
eval(a / b * 100,
a=(M::http:(sum(error_count)) [1h] BY service),
b=(M::http:(sum(request_count)) [1h] BY service),
alias="error_rate")
Calculate CPU Usage¶
// Usage = used / total * 100
eval(used / total * 100,
used=(M::memory:(sum(used_bytes)) [1h] BY host),
total=(M::memory:(sum(total_bytes)) [1h] BY host),
alias="memory_usage_percent")
Calculate Baseline Comparison Growth Rate¶
// Calculate growth rate of current value relative to baseline
eval(current / baseline - 1,
current=(M::sales_current:(sum(amount)) [7d]),
baseline=(M::sales_baseline:(sum(amount)) [7d]),
alias="growth_rate")
Reference Sub-query Fields¶
// Reference specific fields from sub-queries
eval(a.usage / b.total * 100,
a=(M::cpu:(avg(usage) as usage) [1h] BY host),
b=(M::cpu:(avg(total) as total) [1h] BY host),
alias="cpu_percent")
Notes¶
- All sub-query time windows must be compatible
- Grouping dimensions of sub-queries should be consistent
- Fields referenced in expressions use the
name.fieldformat - If there is only one sub-query, you can directly use the field name
Other Functions¶
fill¶
Fills null values in the query result with a specified value.
Recommended Usage:
fillis recommended to be used as an outer function, operating on the entire query result:Although the syntax
fill(avg(usage), 0)is also supported in the Select clause,fillactually fills the result after aggregation is complete, so the outer function form better matches its working mechanism.
Syntax (Outer Function):
Parameters:
DQL_expression: DQL query expressionvalue: Fill value, supports multiple modes:- Specific value: numeric, string, null
LINEAR: Linear interpolationPREVIOUS: Fill with the previous non-null value
Examples:
// Recommended: use as an outer function
fill(M::cpu:(avg(usage)) [1h::5m] BY host, 0)
// Linear interpolation fill
fill(M::cpu:(avg(usage)) [1h::5m] BY host, LINEAR)
// Fill with previous value
fill(M::cpu:(avg(usage)) [1h::5m] BY host, PREVIOUS)
now¶
Returns the current timestamp (in milliseconds).
Syntax:
Examples:
// Query recently updated data
O::HOST:(*) {last_update_time > now() - 600000} // Updated within the last 10 minutes
unwrap¶
Unwraps the wrapper of an aggregation result.
Syntax:
Examples:
Show Functions¶
Show functions are used to view metadata (such as measurements, tags, fields, cardinality, and series count), commonly used for modeling troubleshooting and pre-query exploration.
General Syntax¶
where,time_window,LIMIT,OFFSETare all optional.LIMIT/OFFSETcannot be negative.
Built-in Show Functions in M Namespace¶
| Function | Parameters | Return Columns | Description |
|---|---|---|---|
show_measurement |
Optional re('pattern') |
name |
List measurements |
show_tag_key |
Optional from=['measurement'] |
tagKey |
List tag keys |
show_field_key |
Optional from=['measurement'] |
fieldKey, fieldType |
List field keys (current fieldType is float) |
show_tag_value |
keyin=['tagKey'] (required), optional from |
key, value |
List tag values |
show_measurement_cardinality |
No mandatory parameters | count |
Number of measurements |
show_series_cardinality |
No mandatory parameters | count |
Series cardinality (estimated) |
show_tag_key_cardinality |
No mandatory parameters | count |
Tag key cardinality (estimated) |
show_tag_value_cardinality |
keyin=['tagKey'] (required) |
count |
Value cardinality of a specified tag key (estimated) |
show_field_key_cardinality |
No mandatory parameters | count |
Field key cardinality (estimated) |
show_series_count_by_field_key |
from=['measurement'] (recommended) |
name, count |
Series count by field key |
show_series_count_by_tag_key |
from=['measurement'] (recommended) |
name, count, value_count |
Series count and value count by tag key |
show_series_count_by_tag_value |
keyin=['tagKey'] (required), from=['measurement'] (recommended) |
name, count |
Series count by value of a specified tag key |
Cardinality-related functions use HyperLogLog merge under the hood, returning estimated values.
Show Functions for Non-M Namespaces (Suffix Pattern)¶
For non-M namespaces, the following suffix patterns are supported:
show_<namespace>_sourceshow_<namespace>_classshow_<namespace>_typeshow_<namespace>_fieldshow_<namespace>_label
The <namespace> is automatically mapped from the middle segment of the function name, for example:
show_logging_source->Lshow_tracing_field->Tshow_object_source->O
Common examples:
show_logging_source()
show_tracing_field('mysql')
show_logging_field('*')
show_logging_label(name='env')
show_logging_label(names=['env', 'team'])
Parameters and Behavior Notes¶
from: List of measurements, supports a string or an array of strings.keyin: List of tag keys, supports a string or an array of strings.field: List of fields, supports a string or an array of strings (used for metric show field filtering).- For
show_*_field: - An unnamed parameter (e.g.,
'mysql') is typically used as a source filter. '*'is equivalent to not specifying a source.- Named parameters are converted to where filter conditions.
- For
show_*_label: - Requires named parameters;
namesis treated as an alias forname.
Constraints and Notes¶
show_tag_valueandshow_tag_value_cardinalitymust providekeyin.show_series_count_by_tag_valuemust providekeyin.show_series_count_by_*must providefrom, or an equivalent source constraint in the where clause (e.g.,@__source__condition).show_<namespace>_source,show_<namespace>_class,show_<namespace>_typecurrently share the same execution path, returning a deduplicated list of sources.- The parser currently does not support the
show_<namespace>_indexsyntax (even if the execution layer has a corresponding branch). - When the Query API does not provide a show time range, some log show queries will fall back to the last 30-minute window.
Return Examples¶
The following examples only show typical column structures and sample rows; actual results will be affected by tenant data, filter conditions, time range, and LIMIT/OFFSET.
| Query | Typical Columns | Sample Rows (Illustrative) |
|---|---|---|
show_measurement() |
name |
cpu, disk, memory |
show_tag_value(from=['cpu'], keyin=['host']) |
key, value |
host, web-01; host, web-02 |
show_series_count_by_tag_key(from=['cpu']) |
name, count, value_count |
host, 3200, 120; service, 2800, 35 |
show_tag_value_cardinality(keyin=['host']) |
count |
120 |
show_logging_field('*') |
fieldKey, fieldType, fieldIndices |
service, keyword, ["idx_service"] |
show_logging_source() |
source |
nginx, mysql, redis |
Function Category Quick Reference Table¶
Basic Aggregation¶
| Function | Description | Exact/Estimate |
|---|---|---|
| sum | Sum | Exact |
| avg | Average | Exact |
| count | Count | Exact |
| count_distinct | Distinct count | Estimate (HyperLogLog, error ≈ 0.4%) |
| min / max | Minimum / Maximum | Exact |
| first / last | First / Last value | Exact |
| any | Any value | Exact |
Statistical Aggregation¶
| Function | Description | Exact/Estimate |
|---|---|---|
| percentile / pXX | Percentile | Estimate (log histogram) |
| median | Median | Estimate |
| stddev | Standard deviation | Exact |
| mode | Mode | Exact |
| spread | Range | Exact |
| count_series | Time series count | Exact |
Time Series Analysis Aggregation¶
| Function | Description | Notes |
|---|---|---|
| ewma | Exponentially Weighted Moving Average | Supports [...:ewma(alpha)] |
| slope | Linear trend slope | Supports Rollup shorthand |
| zscore | Latest point Z-Score | Supports Rollup shorthand |
| mad_score | Latest point MAD anomaly score | Supports Rollup shorthand |
| change_score | Series change score | Supports Rollup shorthand |
| corr | Pearson correlation coefficient between two fields | Requires two input fields |
Filtered Aggregation¶
| Function | Description | Exact/Estimate |
|---|---|---|
| top / bottom | TopN / BottomN | Exact |
| count_filter | Conditional count | Exact |
Histogram Functions¶
| Function | Description | Applicable Data Source | Exact/Estimate |
|---|---|---|---|
| histogram_auto | Automatic histogram (Recommended) | Logs, Trace detail data | Estimate |
| histogram | Fixed bucket boundary histogram (Deprecated) | Logs, Trace detail data | Exact |
| histogram_quantile | Calculate quantiles from Prometheus histogram | Prometheus metrics | Estimate |
Collection Functions¶
| Function | Description | Exact/Estimate |
|---|---|---|
| distinct | List of distinct values | Exact |
| distinct_by_collapse | Collapse deduplication (retain other fields) | Exact |
| collect | Collect all values | Exact |
| collect_distinct | Collect distinct values | Exact |
Helper Functions¶
| Function | Description | Exact/Estimate |
|---|---|---|
| default | Set default value | Exact |
Time Series Functions¶
| Function | Description |
|---|---|
| rate | Rate of change (per second) |
| irate | Instant rate of change |
| increase | Increase |
| derivative | Derivative |
| difference | Difference |
| non_negative_derivative | Non-negative derivative |
| non_negative_difference | Non-negative difference |
| moving_average | Moving average |
| cumsum | Cumulative sum |
| ewma | Exponentially Weighted Moving Average |
| slope | Trend slope |
| zscore | Latest point Z-Score |
| mad_score | MAD anomaly score |
| change_score | Change score |
| corr | Correlation coefficient |
Rollup Functions¶
| Function | Description |
|---|---|
rate |
Calculates rate of change (per second) |
irate |
Calculates instant rate of change |
increase |
Calculates increase |
rate_over_sum |
Calculates per-second average |
rate_over_count |
Calculates per-second count |
deriv |
Calculates derivative |
difference |
Calculates difference |
sum |
Sum |
avg |
Average |
min |
Minimum |
max |
Maximum |
count |
Count |
first |
First value |
last |
Last value |
stddev |
Standard deviation |
mode |
Mode |
spread |
Range |
any |
Any value |
ewma(alpha) |
Exponentially Weighted Moving Average |
moving_average(n) |
Moving average |
percentile(p) |
Percentile |
slope |
Linear trend slope |
zscore |
Latest point Z-Score |
mad_score |
Latest point MAD anomaly score |
change_score |
Series change score |
Transformation Functions¶
| Function | Description |
|---|---|
| abs | Absolute value |
| round / ceil / floor | Rounding |
| log / log2 / log10 | Logarithm |
| int / uint / float / string / bool | Type conversion |
| substr | Substring |
| regexp_extract | Regex extraction |
| regexp_extract_all | Extract all regex matches |
| regexp_replace | Regex replacement |
| md5 | MD5 hash |
| concat | String concatenation |
| set | Array deduplication and sort |
| drain | Log clustering |
Matching Functions¶
| Function | Description |
|---|---|
| match | Substring matching |
| phrase / search | Tokenized phrase matching |
| re / regex / regexp | Regex matching |
| wildcard | Wildcard matching |
| cidr | CIDR network matching |
| query_string | Query string syntax |
| exists | Field existence check |
Outer Functions¶
| Function | Description |
|---|---|
| cumsum | Cumulative sum |
| rate / irate | Rate of change (non-negative) |
| deriv | Derivative (allows negative values) |
| increase | Increase (non-negative) |
| difference | Difference (allows negative values) |
| moving_average | Moving average |
| top / bottom | TopN |
| dbscan | DBSCAN outlier detection |
| forecast | Linear trend forecasting |
| fill | Null value filling (recommended as outer) |
| abs / round / ceil / floor | Mathematical operations |
| set | Deduplicate and sort |
| concat | String concatenation |
Expression Calculation¶
| Function | Description |
|---|---|
| eval | Multi-query expression calculation |