Skip to content

Infrastructure Liveness Detection V2

Document Purpose

This document is Step 2 of the detection rule configuration process. After completing the configuration, return to the main document to continue with Step 3: Event Notification.

Data scope: Objects (O). It is used to monitor the stability of data reporting of key infrastructure objects (such as hosts, containers, Pods, etc.). By setting detection conditions and alert levels, you can promptly detect and handle anomalies to ensure stable infrastructure operation.

Detection Configuration

Detection Frequency

Set the time period for running detection.

  • Preset options: 5 minutes, 15 minutes, 30 minutes, 1 hour, 6 hours, 12 hours, 24 hours

  • Crontab mode: click "Switch to Crontab Mode" to configure a custom period

Note

Because object data is reported every 5 minutes, the detection frequency should be greater than 5 minutes and less than 1 day to avoid false positives or detection delays.

Detection Range

Set the time range of data queried for each detection run (must be ≥ the detection frequency).

Detection Frequency Detection Range (Dropdown Options)
5m Last 10 minutes / Last 15 minutes / Last 30 minutes / Last 1 hour / Last 6 hours / Last 12 hours / Last 24 hours
15m Last 15 minutes / Last 30 minutes / Last 1 hour / Last 6 hours / Last 12 hours / Last 24 hours
30m Last 30 minutes / Last 1 hour / Last 6 hours / Last 12 hours / Last 24 hours
1h Last 1 hour / Last 6 hours / Last 12 hours / Last 24 hours
6h Last 6 hours / Last 12 hours / Last 24 hours
12h Last 12 hours / Last 24 hours
24h Last 24 hours
  • Custom format: for example, 20m (last 20 minutes), 2h (last 2 hours), and 1d (last 1 day) are supported.

Detection Metric

Monitors the data reporting status based on infrastructure object data.

Configuration Item Description
Infrastructure Type Select the object type to monitor: Host, Process, Container, Pod, Service, Deployment, Node, ReplicaSet, Job, CronJob
Detection Target Select the detection scope:

  • All: run detection on all objects of this type in the workspace
  • Custom: limit the detection scope using Wildcard fuzzy matching or tag filtering
    		•
    Additional Information After you select fields, the system performs additional queries to enrich the event content, but these fields do not participate in trigger condition evaluation. If multiple matching values are detected, a random record is returned. Supported fields include: unicast_ip, Scheck, instance_id, region, etc.

    Custom Detection Target Configuration

    After selecting "Custom", the following filtering methods are supported:

    • Wildcard matching: enter a wildcard expression for fuzzy matching (for example, web-*). If the content contains the special character "backslash \", it must be escaped to take effect;
    • Tag filtering: filter precisely by tags (such as 操作系统, 标签, etc.) and more fields (datakit_ver, zone_id, cloud_provider, etc.).

    Trigger Conditions

    Configure the trigger conditions for each alert level (Critical, Major, Warning, Info) and the normal recovery condition.

    Level Configuration Description
    Critical The detection target has not reported data for [N] consecutive minutes Highest-level alert, requires immediate handling
    Major The detection target has not reported data for [N] consecutive minutes High-level alert, requires priority handling
    Warning The detection target has not reported data for [N] consecutive minutes Medium-level alert, requires attention
    Info The detection target has not reported data for [N] consecutive minutes Low-level alert, requires attention
    Normal No events generated for [N] consecutive detection runs After an anomaly event is generated, if no anomaly is triggered for N consecutive detection runs, a recovery event (normal event) is generated
    Input Value Range

    The input value range for Critical, Major, Warning, and Info is 5 to 999 minutes. If the input value is less than 5 minutes, adjust the detection frequency or range to avoid false positives.

    Multi-Object Detection Logic

    When the query returns multiple objects, an event at the corresponding level is triggered as long as any one of these objects meets the configured conditions.

    For more details, refer to Event Level Description.

    Subsequent Configuration

    After completing the above detection configuration, continue with the following configuration:

    1. Event Notification: define the event title, content, notification members, data gap handling, and associated incidents;
    2. Alert Configuration: select an alert policy, and configure notification targets and mute periods;
    3. Linking: link a dashboard for quick navigation to view data;
    4. Permissions: set operation permissions to control who can edit or delete this monitor.