Blacklist¶

By setting up a blacklist, you can filter out different types of data that meet specific conditions. Once the blacklist is configured, data that meets the conditions will no longer be reported to the TrueWatch workspace, helping you save on data storage costs.

Prerequisites¶

• Install DataKit;
• If you need to configure data other than logs, the DataKit version must be higher than 1.4.7.

Create Blacklist¶

1. Click Manage > Blacklist > Create Blacklist;
2. Define the name and description of the current blacklist rule;
3. Select the data source type;
4. Add one or more filtering rules;
5. Click Confirm to enable the data blacklist filtering rule.

Data Source¶

The blacklist name is automatically generated based on the data source, including logs, basic objects, Resource Catalog, network, APM, RUM, events, Metrics, and Profile.

After entering the field name, field value, and other information, it will take effect after configuring the data source and fields through DataKit and reporting the data.

Filtering¶

Supports two condition selections: "Any" and "All". "Any" is an "OR" condition, and "All" is an "AND" condition.

• Field Name: Supports manual input of field names, which must be precise values. You can view the field names that need to be matched in the "Show Columns" of the Explorer.

• Field Value: Supports manual input of field values, supports single or multiple values, and supports regular expressions.

• Operators: Supports in / not in / match / not match 4 modes. in / not in is exact matching, and match / not match is regular expression matching.

Note:

• If you only need to create a blacklist for log data, you can go to Logs > Blacklist to configure it.
• Data types support string, integer, and floating-point types;
• If the data source is logs, a log filtering rule will be created in the Logs > Blacklist menu, and vice versa.

Example¶

In the following example, the blacklist is named "Conditional Filtering". Select logs from All Sources, where status is ok or info, host is not hz-dataflux-saas-daily-01, and service does not contain the word kodo. Data that meets all three matching rules will be filtered and will not be reported to the workspace.

After setting up the blacklist, you can check whether the blacklist is effective in the Explorer based on the filtering conditions. Once the blacklist is created and effective, data that meets the filtering conditions will no longer be reported to the workspace.

List Operations¶

You can manage the blacklist list through the following operations:

1. Filter based on different data types;
2. Search for the blacklist name in the search bar to locate it.
3. Modify the created data filtering rules;
4. Delete existing filtering rules. After deletion, data will be reported to the workspace normally.

5. Batch operations: Click to batch export or delete blacklists.

   Note: This feature is only displayed for workspace owners, administrators, and regular members, and is not displayed for read-only members.

6. You can create a blacklist by importing a JSON file, and the imported JSON file must be a configuration JSON file from TrueWatch.

Precautions¶

1. If you configured the blacklist filtering in the datakit.conf file during the DataKit installation and configuration, the blacklist rules configured in TrueWatch will not take effect;

2. DataKit pulls data every 10 seconds, and the blacklist configuration will not take effect immediately. You need to wait at least 10 seconds;

3. After the blacklist configuration is completed, it is uniformly saved in the .pull file in the /usr/local/datakit/data directory of DataKit.

Further Reading¶