Generate Metrics¶

Generate new metric data from existing data within the current workspace, allowing you to design and implement new technical metrics based on actual business needs.

Important Notes¶

1. Only roles with the "Generate Metrics Configuration Management" permission can create and edit metrics.

2. After generation, metrics will be stored according to the current default data storage policy, and charges will be based on the number of generated time series.

3. If no data is reported after generating metrics, they cannot be queried or analyzed within the workspace.

What Problems Can It Solve?¶

Use Case 1: Extract Business Metrics from Logs

Business logs contain key business data (e.g., order amounts, error codes) that need to be transformed into monitorable metrics. By generating metrics, you can select a log data source, configure query conditions to filter target logs, extract fields to generate metrics (e.g., error log count, average response time), and group them by business dimensions (e.g., service, interface, status code).

Use Case 2: APM Data Aggregation

Generate service SLA metrics based on APM trace data. Select the APM data source, group statistics by service and interface, calculate metrics like P99 latency, error rate, throughput, etc., and generate them periodically for continuous monitoring.

Use Case 3: Resource Usage Statistics

Statistics Kubernetes cluster resource usage. Select the Infrastructure or Resource Catalog data source, group by cluster, namespace, node, and count metrics like Pod count, resource requests/limits, etc., to generate resource utilization metrics.

Use Case 4: Multi-source Data Fusion

Combine multiple data sources to generate composite metrics. Create separate generation rules, and generate new metrics based on the generated ones, building a complex business monitoring system.

Scope of Application¶

LOG, APM, RUM, Metrics, Synthetic Tests, Infrastructure, Resource Catalog.

Create¶

1. Select the data source.
2. Configure the data query conditions.
3. Define the generated metric content, setting the method and results for generating metrics, including the generation frequency, the tag names, and the measurement name for the newly generated metrics.

Data Query¶

Metric data additionally supports PromQL queries. For other data types, both simple queries and DQL queries are supported.

For more details, refer to Chart Query.

Aggregation Functions¶

Dimensions¶

Aggregate data according to the selected objects, meaning a statistical value is generated for each selected object in the data request.

Generated Metric Content¶

1. Frequency: The execution cycle for generating metrics. The selected time also serves as the aggregation window. Choosing a frequency of 1 minute means metrics are aggregated and generated every 1 minute, with each aggregation covering a 1-minute time range.

   • 1 minute (default, meaning new metric data is generated every 1 minute)
   • 5 minutes (default selected when the data type is "Infrastructure" or "Resource Catalog")
   • 15 minutes

2. Select the data delay window, specifying the waiting period for the calculation task after the task window ends.

3. Measurement: Set the name of the measurement where the metrics will be stored.

4. Metric: Set the name of the metric. Metric names can be duplicated, and multiple metrics can be added.

5. Tags: Automatically generated based on the dimensions selected in the query.

6. Unit: Optional. Set the unit for the metric. Once set for a generated metric, it can be applied in chart queries.

7. Description: Optional. Set a description for the metric. Once set for a generated metric, it can be applied in chart queries.

After completing the form, click OK to finish creating the metric generation rule and start data collection.

Data Delay Window¶

The duration the system continues to wait for data arrival after the task window ends. Enabling this can accommodate data reporting delays and improve metric accuracy.

When generating metrics, the system executes queries periodically according to the frequency you set (e.g., 1 minute). Each query retrieves data from a fixed time range (the "task window"). However, due to delays between data generation and reporting/storage, some data may arrive after the task window ends, causing these "late" data points to be missed in the statistics.

Enabling the data delay window allows the system to wait for an additional period after the task window ends, giving delayed data a chance to be included in the statistics, thereby improving metric accuracy.

1. It is recommended to enable the data delay window in the following scenarios:

2. Calculation Logic Example (Frequency: 1 minute, Data Delay Window: 1 minute):

Task execution time: 13:01:00
Task window (actual query time range): 13:00:00 ~ 13:01:00
Data delay window: 1 minute
Actual query execution time: 13:02:00 (after waiting 1 minute)

That is: The system executes the query at 13:02:00, retrieving data with time between 13:00:00 ~ 13:01:00. Even if some data is stored after 13:01:00, as long as its time belongs to 13:00:00 ~ 13:01:00, it will be counted.

Manage List¶

• Edit: View all created metric generation rules and edit them.

• Enable/Disable: Modify the rule's status. When a metric generation rule is disabled, the corresponding data will not be written to the measurement. Writing resumes after enabling.

• Delete: Delete unnecessary rules. After a rule is deleted, the measurement is not deleted, but data writing stops.

• Batch operations: Perform batch operations on specific rules, including enabling, disabling, deleting, and exporting rules.

• Import: Import metric generation data.

• View Metrics

  • View in Metric Explorer: Jump to the Metric Explorer page for query and analysis.
  • View in Metric Management: Jump to the Metric Management page to view metrics and tags. You can edit metric units and descriptions.

Application Scenarios¶

Chart Query¶

Query and analyze metric data in Visualization Charts.

Query Tool¶

Query and analyze metric data in Shortcut > Query Tool > DQL Query.