Interval Detection¶
Current Document Positioning
This document is the second step in the detection rule configuration process. After configuration, please return to the main document to continue with step three: Event Notifications.
Within the selected time range, the system will perform anomaly detection on metrics data. If the proportion of mutation anomalies among the detected data points exceeds the preset threshold percentage, an interval anomaly event will be triggered.
Suitable for monitoring data/metrics with stable trends. For example, when detecting the proportion of data points with CPU usage mutation anomalies in the last 1 day exceeds 10%, an anomaly event is generated.
Detection Configuration¶
Detection Frequency¶
Set the time period for executing detection, automatically matching the selected detection interval.
| Detection Interval (Dropdown Options) | Detection Frequency |
|---|---|
| 15m | 5m |
| 30m | 5m |
| 1h | 15m |
| 4h | 30m |
| 12h | 1h |
| 1d | 1h |
Detection Interval¶
Set the time range for each detection query (❗️The detection interval should be greater than or equal to the detection frequency, and must match the actual data reporting cycle to avoid missed detections or false alarms).
-
Preset options: Last 15 minutes, Last 30 minutes, Last 1 hour, Last 4 hours, Last 12 hours, Last 1 day
-
Custom format: Custom input of detection interval, e.g., 20m (last 20 minutes), 2h (last 2 hours), 1d (last 1 day).
Detection Metrics¶
Define the detection data source and aggregation method based on DQL (❗️Avoid selecting high-cardinality fields as detection dimensions. If configured improperly with overly loose trigger conditions, frequent alerts may occur. The current query maximum return count is 100,000 records).
Configuration Elements¶
| Configuration Item | Description |
|---|---|
| Workspace | Default to the current workspace, can be switched to other authorized workspaces After authorization, you can use the detection metrics of other workspaces under the current account to create monitors. Once the rule is created successfully, cross-workspace alert configuration is achieved. Note that when you select another workspace, the detection metrics dropdown list will only display the data types currently authorized for use in the current workspace. |
| Data Source Type | Metrics, Logs, Infrastructure, Resource Catalog, Events, APM, RUM, Network, and Profile, etc. |
| Query Method | Simple Query, Expression Query |
| Detection Dimensions | Any string type (keyword) field in the configuration data can be selected as a detection dimension. Currently, up to three fields are supported for detection dimensions. By combining multiple detection dimension fields, a specific detection object can be determined. The system will evaluate whether the statistical metric corresponding to the detection object meets the trigger condition threshold. If the condition is met, an event is generated.(For example, selecting detection dimensions host and host_ip, the detection object could be {host: host1, host_ip: 127.0.0.1}). |
| Filter Conditions | Filter the data of the detection metric based on the labels of the metric, limiting the data range for detection. Supports adding one or more label filters. Supports fuzzy match and fuzzy not match filter conditions. |
| Aggregation Algorithm | Avg by (average), Min by (minimum), Max by (maximum), Sum by (sum), Last (last value), First by (first value), Count by (data point count), Count_distinct by (distinct count), p50 (median), p75 (75th percentile), p90 (90th percentile), p99 (99th percentile), etc. |
| Alias | Custom detection metric name |
Click to view Query Method Details.
Trigger Conditions¶
Configure trigger conditions and severity levels. When the query result has multiple values, if any value meets the trigger condition, an event is generated.
Supports configuring four severity levels: Critical, Error, Warning, Info, and a OK recovery condition.
| Level | Configuration | Description |
|---|---|---|
| Critical | When the mutation direction is Up or Down / Up / Down, Result >= [value] % |
Compare the proportion of data points with mutation anomalies. If not within the configured range, an event is triggered. |
| Error | When the mutation direction is Up or Down / Up / Down, Result >= [value] % |
Compare the proportion of data points with mutation anomalies. If not within the configured range, an event is triggered. |
| Warning | When the mutation direction is Up or Down / Up / Down, Result >= [value] % |
Compare the proportion of data points with mutation anomalies. If not within the configured range, an event is triggered. |
| Info | When the mutation direction is Up or Down / Up / Down, Result >= [value] % |
Compare the proportion of data points with mutation anomalies. If not within the configured range, an event is triggered. |
| OK | [N] consecutive detections with no events generated |
After the detection rule takes effect, if within the configured custom number of detections, the data detection result changes from abnormal (Critical, Error, Warning, Info) to normal, a recovery alert event is triggered. ❗️ Recovery alert events are not subject to Alert Silence. If the number of recovery alert event detections is not set, the alert event will not recover and will always appear in the Events > Unrecovered Events List. |
For more details, refer to Event Level Description.
Bulk Alert Protection¶
Enabled by default.
When the number of alerts generated by a single detection exceeds the preset threshold, the system automatically switches to a summary-by-status strategy: Instead of processing each alert object individually, a small number of summary alerts are generated based on the event status and pushed.
This ensures timely notification while significantly reducing alert noise, avoiding timeout risks due to processing too many alerts.
When this switch is enabled, subsequent event details generated by the monitor after detecting anomalies will not display history records or related events.
Data Gap¶
The handling strategy when the query result of the detection metric is empty within the detection interval:
| Option | Description |
|---|---|
| Do not trigger events (default) | Linked with the detection interval time range, determines whether to generate an event based on the query result of the detection metric in the last few minutes. Suitable for scenarios where data absence is allowed. |
| Treat query result as 0 | Linked with the detection interval time range, treats the query result of the detection metric in the last few minutes as 0, and re-compares it with the threshold configured in the Trigger Conditions above to determine whether to trigger an anomaly event. |
| Custom fill and trigger events | Supports custom filling of the detection interval value, and triggers the following event types separately: Data Gap Event, Critical Event, Warning Event, Info Event, and Recovery Event. ❗️When selecting this strategy, it is recommended to configure the custom data gap time ≥ the detection interval time interval. If the configured time is ≤ the detection interval time interval, both data gap and anomaly may be satisfied simultaneously, in which case the data gap handling result will be applied first. |
When configuring trigger conditions, data gap, and information generation simultaneously, the priority for triggering is: Data Gap > Trigger Conditions > Information Event Generation.
That is: first determine whether there is a data gap, then determine whether the threshold is triggered, and finally determine whether to generate an information event.
Information Generation¶
When this option is enabled, the Information Generation Condition must be configured. The system will only write an "Information" event when the detection result does not trigger any of the "Critical", "Error", "Warning", or "Info" thresholds and meets the information generation condition.
Suitable for scenarios where normal status changes or low-priority information needs to be recorded.
Subsequent Configuration¶
After completing the above detection configuration, proceed to configure:
-
Event Notifications: Define the event title, content, notified members, data gap handling, and related faults.
-
Alert Configuration: Select the alert strategy, set notification targets and mute periods.
-
Linkage: Link to dashboards for quick navigation to view data.
-
Permissions: Set operation permissions to control who can edit/delete this monitor.