Process Anomaly Detection¶
Document Position
This document is the second step in the detection rule configuration process. After completing the configuration, please return to the main document to continue with the third step: Event Notification.
Used to monitor process data in the workspace, supports configuring alert trigger conditions for one or more field types in process data. By monitoring the count of processes, it detects process anomalies (such as process disappearance, abnormal increase in process count, etc.) in a timely manner, ensuring the stability of system services.
Data scope: Supports Process data type.
Suitable for scenarios where the running status of specific processes needs to be monitored. For example:
- Monitor whether key business processes (such as
nginx,mysql,java, etc.) are running normally - Monitor whether the number of processes under a specific pattern is abnormal (e.g., too many zombie processes)
- Monitor whether scheduled task processes start normally
Detection Configuration¶
Detection Frequency¶
Set the time interval for executing detection.
-
Preset options: 1 minute, 5 minutes (default), 10 minutes, 15 minutes, 30 minutes, 1 hour
-
Crontab mode: Click "Switch to Crontab mode" to configure a custom period. Supports configuring scheduled task execution based on seconds, minutes, hours, days, months, weeks, etc.
Detection Interval¶
Set the data time range queried for each detection (❗️Detection interval should be greater than or equal to the detection frequency, and must match the actual data reporting cycle to avoid missed or false alarms).
| Detection Frequency | Detection Interval (dropdown options) |
|---|---|
| 30s | 1m/5m/15m/30m/1h/3h |
| 1m | 1m/5m/15m/30m/1h/3h |
| 5m | 5m/15m/30m/1h/3h |
| 15m | 15m/30m/1h/3h/6h |
| 30m | 30m/1h/3h/6h |
| 1h | 1h/3h/6h/12h/24h |
| 6h | 6h/12h/24h |
| 12h | 12h/24h |
| 24h | 24h |
- Custom format: Custom input for detection interval, e.g., 20m (last 20 minutes), 2h (last 2 hours), 1d (last 1 day).
Detection Object¶
Set the process data to be detected (❗️Avoid selecting high-cardinality fields as detection dimensions. Improper configuration with overly loose trigger conditions may cause frequent alerts. The maximum number of records returned by a single query is 100,000).
Supports setting the number of occurrences of one or more field type keywords in the process data of the current workspace within a specified time range.
Configuration Elements¶
| Configuration Item | Description |
|---|---|
| Detection Type | Fixed as "Process Count Statistics", used to count the number of processes matching the conditions |
| Process | Manually enter the process name. Supports wildcards for fuzzy matching (e.g., k8s*, C:\\Windows\\*). Special characters do not need escaping. Multiple values are separated by "," |
| Filter Conditions | Supports filtering by fields of process data to limit the data range for detection. Supports adding one or more tag filters. Supports fuzzy matching and fuzzy non-matching filter conditions |
| Detection Dimensions | Supports selecting any string type (keyword) field in the data as a detection dimension. Currently, up to three dimensions can be selected. By combining multiple detection dimension fields, a specific detection object can be determined. The system will determine whether the statistical indicator corresponding to a detection object meets the trigger condition threshold. If the condition is met, an event is generated.(For example, if detection dimensions host and host_ip are selected, the detection object can be {host: host1, host_ip: 127.0.0.1}.) |
Trigger Conditions¶
Configure trigger conditions and severity levels. When the query result has multiple values, an event is generated if any value meets the trigger condition.
Supports configuring Critical, Major, Warning, Info four-level thresholds, and OK recovery conditions.
| Level | Configuration | Description |
|---|---|---|
| Critical | When Result >= [value] |
Highest level alert, requires immediate handling |
| Major | When Result >= [value] |
High level alert, requires priority handling |
| Warning | When Result >= [value] |
Medium level alert, requires attention |
| Info | When Result >= [value] |
Low level alert, requires notice |
| OK | [N] consecutive detections with no events generated |
After the detection rule takes effect, if the data detection result returns to normal (from Critical, Major, Warning, Info) within the configured number of custom detection counts, a recovery alert event is triggered. ❗️ Recovery alert events are not limited by Alert Silence. If no recovery alert event detection count is set, the alert event will not recover and will remain in the Events > Unrecovered Events List |
For more details, refer to Event Level Description.
Advanced Options¶
Continuous Trigger Judgment¶
When enabled, events are generated only when the trigger condition is continuously met, avoiding false alarms caused by transient fluctuations (❗️Maximum configuration limit is 10 times).
Mass Alert Protection¶
Enabled by default.
When the number of alerts generated by a single detection exceeds the preset threshold, the system automatically switches to a summary-by-status strategy: Instead of processing each alert object individually, a small number of summary alerts are generated based on event status and pushed.
This ensures notification timeliness while significantly reducing alert noise and avoiding timeout risks due to processing too many alerts.
When this switch is enabled, related event details generated by subsequent monitor detections will not display historical records or associated events.
Data Gap¶
The handling strategy when the query result of the detection metric is empty within the detection interval:
| Option | Description |
|---|---|
| Do not trigger events (default) | Linked to the detection interval time range, determines whether to generate events based on the query result of the detection metric within the last few minutes. Suitable for scenarios where data loss is acceptable |
| Treat query result as 0 | Linked to the detection interval time range, treats the query result of the detection metric within the last few minutes as 0, and re-compares it with the threshold configured in the Trigger Conditions above to determine whether to trigger an anomaly event |
| Custom fill and trigger events | Supports custom filling of the detection interval value and separately triggering the following event types: Data Gap Event, Critical Event, Major Event, Warning Event, Info Event, and Recovery Event. ❗️When selecting this strategy, it is recommended that the custom data gap time configuration be ≥ the detection interval time interval; if the configured time is ≤ the detection interval time interval, both data gap and anomaly may be satisfied simultaneously, in which case the data gap processing result will be applied first |
When trigger conditions, data gap, and info generation are configured simultaneously, the priority for judgment is: Data Gap > Trigger Conditions > Info Event Generation.
That is: first determine whether there is a gap, then determine whether the threshold is triggered, and finally determine whether to generate an info event.
Info Generation¶
When this option is enabled, Info Generation Conditions must be configured. The system will only write an "Info" event when the detection result does not trigger any of the "Critical", "Major", "Warning", or "Info" thresholds and meets the info generation conditions.
Suitable for scenarios where normal state changes or low-priority information needs to be recorded.
Subsequent Configuration¶
After completing the above detection configuration, continue to configure:
-
Event Notification: Define event title, content, notification members, data gap handling, and associated faults;
-
Alert Configuration: Select alert strategy, set notification targets and silence periods;
-
Association: Associate with dashboards for quick navigation to view data;
-
Permissions: Set operation permissions to control who can edit/delete this monitor.