Data Forwarding to Kafka Message Queues¶
- Complete the rule name and filter condition settings;
- Continue to configure archiving.
Configure¶
-
Address:
Host:Port, with multiple nodes separated by commas. -
Message topic: the Topic name.
-
Partition Key: optional. Below the message topic, select or enter a field name. Multiple fields can be added. The system uses the values of the selected fields to generate the partition key. For details, see Partition Key.
-
Security protocol: On the Kafka side, SASL can use PLAINTEXT or SSL as the transport layer, corresponding to the SASL_PLAINTEXT or SASL_SSL security protocols. If the SASL_SSL security protocol is used, an SSL certificate must be configured.
-
Click Test Connection. If the above information meets the requirements, a prompt indicates that the connection test succeeded. Click OK to save the current rule.
If the test fails:
You need to verify:
- Whether the address is correct
- Whether the message topic name is correct
- Whether the SSL certificate is correct
- Whether the username is correct
- Whether the password is correct
Partition Key¶
The partition key determines which partition data is written to in a message topic (Topic). When configuring, enter a field name from your data. The system reads the value of the corresponding field in each data record to generate the key. For example, after selecting userid, the userid value of each record is used, rather than using the field name userid as a fixed key for all data.
| Configuration Method | Partition Description |
|---|---|
| Single field | For example, if you select userid, data from the same user enters the same partition, making it easier to process data by user. |
| Multiple fields | The combined values of the selected fields are used to generate the key. For example, if you select userid and region, two records with the same values for both fields will enter the same partition. |
| Not configured | The default partitioning method is used. |
Note
Data with the same key enters the same partition. This applies when the same Topic is used and the partition count, partition key configuration, and partitioning rules remain unchanged. After these configurations are adjusted, new data with the same key may enter other partitions.
PLAINTEXT¶
No security authentication is required. You can test the connection directly.
SASL_PLAINTEXT¶
The authentication method defaults to PLAIN, with SCRAM-SHA-256 and SCRAM-SHA-512 as alternatives.
Enter the username/password used for security authentication on the Kafka side, then test the connection.
SASL_SSL¶
Here, you need to upload the SSL certificate.
The authentication method defaults to PLAIN, with SCRAM-SHA-256 and SCRAM-SHA-512 as alternatives.
Enter the username/password used for security authentication on the Kafka side, then test the connection.


