Threshold Detection¶

Data range: Supports all data types, such as Metrics M, Logs L, Traces T, RUM data R, Objects O, etc.

Detection Configuration¶

Detection Frequency¶

Sets the time period for executing detection.

• Preset options: 1 minute, 5 minutes, 10 minutes, 15 minutes, 30 minutes, 1 hour

• Crontab mode: Click "Switch to Crontab mode" to configure custom periods, supporting scheduled tasks based on seconds, minutes, hours, days, months, weeks, etc.

Detection Interval¶

Sets the data time range for each detection query (❗️The detection interval should be greater than or equal to the detection frequency, and must match the actual data reporting period to avoid missed or false alarms).

• Preset options:

• Custom format: Custom input detection interval, e.g., 20m (last 20 minutes), 2h (last 2 hours), 1d (last 1 day).

Detection Metrics¶

Defines the detection data source and aggregation method based on DQL.

Avoid selecting high-cardinality fields as detection dimensions. Improper configuration with overly lenient trigger conditions may cause frequent alerts. The current query returns a maximum of 100,000 records.

Configuration example: When configuring detection metrics for a monitor, trace_id was selected as the grouping field.

The system will then prompt you:

The current grouping field trace_id is a high-cardinality risk field. Using such fields for aggregation will generate a massive number of groups, which can easily lead to an alert explosion and cause the monitor to be automatically suspended by the system. Please optimize the grouping settings.

Configuration Elements¶

Click to view Query Method Details, More Function Usage Instructions.

Trigger Conditions¶

Configures trigger conditions and severity levels. When the query result has multiple values, an event is generated if any value meets the trigger condition.

Supports configuring Critical, Error, Warning, Info four-level thresholds, and OK recovery conditions.

For more details, refer to Event Level Description.

Advanced Options¶

Continuous Trigger Judgment¶

When enabled, events are generated only when the trigger condition is continuously met, avoiding false alarms from transient fluctuations (❗️Maximum configuration limit is 10 times).

Large-Scale Alert Protection¶

Enabled by default.

When the number of alerts generated by a single detection exceeds the preset threshold, the system automatically switches to a status-based aggregation strategy: Instead of processing each alert object individually, a small number of summary alerts are generated based on event status and pushed.

This ensures timely notifications while significantly reducing alert noise and avoiding timeout risks due to processing too many alerts.

When this switch is enabled, subsequent Event Details generated by the monitor after detecting anomalies will not display historical records or related events.

Recovery Conditions¶

Configures recovery conditions and severity levels. When the query result has multiple values, a recovery event is generated if any value meets the trigger condition.

Set independent recovery thresholds for different levels to achieve downgraded recovery. For example: a Critical alert needs to recover when the value drops below 70, while a Warning alert can recover when below 80.

Recovery Alert Logic¶

When "Recovery Conditions" is enabled, the system uses the Fault ID as a unique identifier to manage the entire lifecycle of the alert (including creating Issues, etc.).

When the graded recovery function is enabled simultaneously:

• The platform configures a separate set of recovery rules (i.e., recovery thresholds) for each alert level (e.g., critical, warning)

• The alert status and recovery status of each level are calculated independently

• It does not affect the original alert lifecycle identified by the Fault ID

Therefore, when the monitor triggers an alert for the first time (i.e., starts a new alert lifecycle), the system generates two alert messages simultaneously. They appear similar because:

1. The first alert source: Overall detection (check), representing the start of the entire fault lifecycle (based on the original rule);

2. The second alert source: Graded detection (critical/error/warning/...), indicating that the graded recovery function has been activated, used to present the specific alert level and its subsequent recovery status (e.g., critical_ok).

In the above, the df_monitor_checker_sub field is the core basis for distinguishing the two types of alerts:

• check: Represents the result of the overall detection;

• Other values (e.g., critical, error, warning, etc.): Correspond to the results of the graded detection rules.

Therefore, when the alert is triggered for the first time, two records appear with similar content but different sources and purposes.

Data Gap¶

Handling strategy when the query result of the detection metric is empty within the detection interval:

When Trigger Conditions, Data Gap, and Info Generation are configured simultaneously, the priority for judgment is: Data Gap > Trigger Conditions > Info Event Generation.

That is: first determine if there is a data gap, then determine if the threshold is triggered, and finally determine if an info event is generated.

Info Generation¶

When this option is enabled, you need to configure Info Generation Conditions. An "Info" event is written only when the detection result does not trigger any of the "Critical", "Error", "Warning", or "Info" thresholds, and the info generation condition is met.

Suitable for scenarios where normal state changes or low-priority information needs to be recorded.

Subsequent Configuration¶

After completing the above detection configuration, proceed to configure:

1. Event Notification: Define the event title, content, notification members, data gap handling, and associated faults;

2. Alert Configuration: Select alert strategies, set notification targets and mute periods;

3. Association: Associate dashboards for quick navigation to view data;

4. Permissions: Set operation permissions to control who can edit/delete this monitor.