Create Alert Strategy¶

Create¶

1. Define the name of the current alert strategy;
2. Enter a description for the strategy as needed;
3. Select the associated monitors;
4. Choose the notification time zone;
5. Select an alert strategy that triggers notifications based on level or member;
6. Choose the time range for repeated alerts (original alert silence);
7. Select the alert aggregation mode as needed to determine the final aggregation form of alert notifications;
8. Add operation permissions to the strategy rules as needed;
9. Save to successfully create.

Association¶

On the configuration page, you can click to select the monitoring rules associated with the current alert strategy, including:

• All
• Monitors
• Intelligent Monitoring
• SLO
• Security Monitoring

Here, you can quickly create new monitoring rules as needed.

Configure Notification Rules¶

Currently, two types of notification configurations are supported: based on level and based on member.

The former sets notification targets for abnormal events of a certain level after selecting the event level. If filter conditions are set, the data range of events under a certain level is further limited, and notifications are finally sent to the targets.

The latter first selects members or teams, defines the range of event data they need to focus on or be responsible for, and then within this data range, defines the event level and the corresponding notification targets, achieving a strong association between events and targets.

Level-based Notification Configuration¶

Define notification targets for alerts of each level.

1. Select event level.

   • One event level can be selected multiple times;
   • Based on the selected event level, you can link alert aggregation.

2. Select the notification targets for events of this level.

Member-based Notification Configuration¶

Configuring notification rules based on members enables precise point-to-point alert notifications. At the same time, in one alert rule, you can configure different notification ranges, levels, and methods for multiple groups of members, and customize the notification time range for multiple groups of members.

1. Define the name of the notification rule;
2. Select the members and teams to be notified;
3. Add filter conditions to achieve tag matching;
4. For the filtered event data, you can set corresponding notification targets for different event levels;
5. Enable the custom notification time range configuration as needed.

Add Filter Conditions¶

Whether configuring notifications based on level or member, adding specific filter conditions can:

• For level-based notifications, further refine the data range of events of a specific level;
• For member-based notifications, limit members or teams to only focus on events that match specific tags.

After adding filters, only events that meet the level requirements and filter conditions will trigger notifications.

After clicking the filter button, the system will automatically fetch the fields of the current workspace and set filter conditions in the form of key:value. You can choose the following matching methods: equal, not equal, wildcard, wildcard negation, and regular expression matching. Multiple filter conditions with the same key field are in OR relationship, and filter conditions with different key fields are in AND relationship.

You can configure filter conditions in the following two ways:

• Directly select fields and set conditions on the page.
• Write regular expressions to achieve more complex filtering logic, meeting fine-grained configuration requirements.

Alert Escalation Notification¶

If a monitor frequently detects anomalies of the same level in a short period of time, it may indicate a persistent issue. At this time, other notification targets may be needed to resolve this type of issue. You can adopt the method of adding escalation notification rules. This way, when anomalies persist, the system will automatically escalate them to urgent notifications and send them to designated recipients, ensuring that issues are promptly noticed and handled.

If a notification rule is configured with two escalation notifications, then:

• When alerts of the same level continue to occur, the system will check the time interval to determine whether to send the first escalation notification;
• After sending the first escalation notification, the system will determine whether to send the second escalation notification based on the time interval configured in the second escalation notification.

Custom Notification Time¶

The scenarios discussed above mainly revolve around the immediacy of automatically triggering notifications when anomalies are detected. However, you can also set specific times for notifications to be sent as needed.

1. Modify the configuration name as needed;

2. Divide the event cycle by day, week, month, and custom dimensions;

   • If custom is selected, you need to upload a CSV file, and the system will automatically fill in the dates filled in the file. The date format in the file should be year/month/day & YYYY/MM/DD; the number of dates in the file should not exceed 365.

3. Limit the time when events occur on the day according to the cycle, and send notifications according to the selected time interval, such as selecting 09:00 - 10:00. When the strategy takes effect, abnormal events generated within this hour will match and flow into this custom configuration;

4. After completing the cycle and time-related configurations, you can select the alert level and notification targets.

Repeated Alerts¶

After setting repeated alert notifications, within a certain time range, event data will continue to be generated, but alert notifications will no longer be sent, and the generated data records will be stored in the event viewer.

Configure Notification Aggregation Rules¶

No Aggregation¶

Default configuration; in this mode, alert events will be merged into one notification every 20 seconds and sent to the corresponding notification targets.

Rule Aggregation¶

In this mode, you can choose the following four aggregation rules and send alert notifications based on the aggregation cycle:

Aggregation Rule	Description
All	Based on the level dimension configured in the alert strategy, generate corresponding alert notifications within the selected aggregation cycle.
Monitors/Intelligent Inspection/SLO	Generate corresponding alert notifications based on the unique ID of monitors, intelligent inspection detection rules, or SLO, linked to the aggregation cycle.
Detection Dimension	Generate corresponding alert notifications based on the detection dimension linked to the aggregation cycle, such as host.
Tags	Multiple selections; can link global tags with monitors, and generate corresponding alert notifications according to the aggregation cycle. If an event has multiple tag values, it will match the corresponding alert notification according to the tag order configured on the page, and the relationship between multiple tag values is OR.

Trigger Strategy¶

In rule aggregation mode, if "Send First Alert" is checked here, it means that df_status will be additionally attached to all, monitors/intelligent inspection/SLO, detection dimensions, and tags, and alerts will be sent externally to avoid missing important abnormal events while waiting for aggregation.

Intelligent Aggregation¶

In this mode, events generated within the aggregation cycle will be clustered into groups based on the selected title or content, and each group will generate one alert notification.

AI Aggregation¶

Using the TrueWatch large model, newly added events can be aggregated into one alert within the set number of minutes, and a new one will be automatically generated after timeout to avoid repeated disturbances.

Aggregation Cycle¶

In rule aggregation and intelligent aggregation modes, you can manually set a time range (1-30 minutes).

Within this time range, newly added events will be aggregated into one alert notification and sent. If the aggregation cycle is exceeded, newly added events will be aggregated into a new alert notification.

Set Operation Permissions¶

After setting the operation permissions of the alert strategy, the roles, team members, and workspace users of your current workspace will perform corresponding operations on the alert strategy according to the assigned permissions. This ensures that different users perform operations that conform to the configuration according to their roles and permission levels.

• Do not enable this configuration: follow the default permissions of "Alert Strategy Configuration Management";
• Enable this configuration and select custom permission objects: at this time, only the creator and the objects granted permissions can enable/disable, edit, and delete the rules set by this alert strategy;
• Enable this configuration but do not select custom permission objects: only the creator has the permissions to enable/disable, edit, and delete this alert strategy.

Further Reading¶