Skip to content

Direct Write Index

Direct Write Index allows specifying the target index for data writing directly during the data collection phase. Unlike log indexes which automatically match based on filter conditions, Direct Write Index requires the DataKit collector to explicitly specify the index name when reporting data.

How It Works

  • The DataKit collector specifies the target index name through configuration when collecting data.
  • After data is reported to the workspace, the system routes it directly to the corresponding index based on the index name.
  • If the index name fails to match, the data will be discarded.

Use Cases

Direct Write Index is suitable for the following scenarios:

  • Deterministic Routing: Ensuring data from specific sources enters a designated index, unaffected by the order of filter rules.
  • High-Performance Writing: Avoiding the overhead of filter condition matching by routing directly to the target index.
  • Multi-Tenant Isolation: Different business lines clearly separate data storage locations through DataKit configuration.

Create

  1. Navigate to the Create > Direct Write Index page.
  2. Customize the name of the index (the DataKit collector will use this name to report data).
  3. Add a description as needed.
  4. Configure the data retention policy: Select the log retention period, after which data will be automatically cleaned up.

  5. Configure Matching Services for APM correlation queries (optional).

    • Supports wildcard fuzzy matching (e.g., service*). Multiple values should be separated by English commas. Once configured, when viewing related logs on the APM details page, the system will prioritize matching logs from the index corresponding to the service.

  6. Confirm.

Notes

  1. Name Consistency: The index name configured in DataKit must exactly match (case-sensitive) the index name created in the console. Otherwise, data will be discarded.
  2. Create Before Use: If the Direct Write Index is not created in the console, data will be discarded even if DataKit is configured.
  3. No Filter Matching: Direct Write Index does not configure filter conditions. Routing relies entirely on the index name specified by DataKit during data reporting.

Further Reading