OpenTelemetry Rust SDK¶

This guide uses SDK instrumentation: initialize the SDK in your application code, create Spans, and configure an Exporter to send traces through DataKit to TrueWatch. This is not a zero-code instrumentation approach; installing dependencies or setting environment variables will not automatically instrument all framework calls. This guide enables only tracing and does not cover Kubernetes.

Rust + OpenTelemetry SDK -> OTLP/HTTP -> DataKit -> TrueWatch

Prerequisites¶

• Install the Rust stable toolchain and Cargo. The example below pins OpenTelemetry 0.31.0; use the current stable toolchain and include Cargo.lock in application version control.
• The example uses synchronous main, a blocking HTTP client, and a background batch export thread; Tokio is not required.
• DataKit is installed, and the reporting endpoint and token have been configured using the installation command of the target Workspace. The application can reach the DataKit HTTP port 9529.

1. Enable the OpenTelemetry Collector¶

On the DataKit host, enter the configuration directory. Copy the sample file only when the configuration does not exist; if the file already exists, adjust it directly:

cd /usr/local/datakit/conf.d/opentelemetry
sudo cp -n opentelemetry.conf.sample opentelemetry.conf

Ensure opentelemetry.conf contains the following configuration. Custom tags are retained through customer_tags:

[[inputs.opentelemetry]]
  customer_tags = ["team", "app.operation"]

  [inputs.opentelemetry.http]
    http_status_ok = 200
    trace_api = "/otel/v1/traces"
    metric_api = "/otel/v1/metrics"
    logs_api = "/otel/v1/logs"

For local access, use 127.0.0.1:9529. For cross-host access, set a listen address reachable by the application in [http_api].listen in the DataKit main configuration datakit.conf, and restrict the network access scope. The HTTP listen address is not set in the collector file.

Restart and check DataKit:

sudo datakit service restart
curl http://127.0.0.1:9529/v1/ping

/v1/ping only verifies that the HTTP service is reachable; it does not mean traces have been ingested. See OpenTelemetry Collector for full instructions. DataKit handles Workspace authentication; the sample application does not configure the Workspace token directly.

2. Instrument the Application with OpenTelemetry¶

Install Dependencies¶

Create a sample project in an empty directory, then set Cargo.toml to the following content. For an existing project, merge the dependencies and do not overwrite the original configuration:

cargo new otel-rust-demo
cd otel-rust-demo

[package]
name = "otel-rust-demo"
version = "0.1.0"
edition = "2021"

[dependencies]
opentelemetry = { version = "=0.31.0", default-features = false, features = ["trace"] }
opentelemetry_sdk = { version = "=0.31.0", default-features = false, features = ["trace"] }
opentelemetry-otlp = { version = "=0.31.0", default-features = false, features = ["trace", "http-proto", "reqwest-blocking-client"] }

Initialize the SDK and Create Spans¶

Save the following content as src/main.rs. The SDK is initialized once at startup, a child Span db.lookup is created under checkout, and the program waits for batch export before exiting:

use opentelemetry::{
    global,
    trace::{TraceContextExt, Tracer},
    KeyValue,
};
use opentelemetry_otlp::{Protocol, SpanExporter, WithExportConfig};
use opentelemetry_sdk::{
    propagation::TraceContextPropagator,
    trace::{Sampler, SdkTracerProvider},
    Resource,
};

fn main() -> Result<(), Box<dyn std::error::Error>> {
    let exporter = SpanExporter::builder()
        .with_http()
        .with_protocol(Protocol::HttpBinary)
        .build()?;

    let provider = SdkTracerProvider::builder()
        .with_batch_exporter(exporter)
        .with_resource(Resource::builder().build())
        .with_sampler(Sampler::ParentBased(Box::new(
            Sampler::TraceIdRatioBased(1.0),
        )))
        .build();

    global::set_text_map_propagator(TraceContextPropagator::new());
    global::set_tracer_provider(provider.clone());
    let tracer = global::tracer("otel-rust-demo");

    tracer.in_span("checkout", |_cx| {
        tracer.in_span("db.lookup", |cx| {
            cx.span().set_attribute(KeyValue::new("app.operation", "lookup"));
        });
    });

    provider.shutdown()?;
    Ok(())
}

Build and Run¶

Configure the following parameters in the same terminal from which you start the application. For cross-host access, replace 127.0.0.1 with the actual DataKit address:

export OTEL_SERVICE_NAME="order-service"
export OTEL_RESOURCE_ATTRIBUTES="deployment.environment.name=prod,service.version=1.0.0,team=backend"
export OTEL_EXPORTER_OTLP_TRACES_ENDPOINT="http://127.0.0.1:9529/otel/v1/traces"

cargo run

The first run downloads and compiles dependencies; afterwards, you can run cargo build --release to build a production binary. Long-running services should keep the Provider and call shutdown() after stopping accepting requests and finishing in-flight Spans.

3. Data Reporting Parameters¶

This example selects HTTP/Protobuf in code with .with_http() and Protocol::HttpBinary; changing OTEL_EXPORTER_OTLP_PROTOCOL will not automatically switch this example to gRPC. To use gRPC, enable grpc-tonic, switch to .with_tonic(), and provide a suitable Tokio runtime.

Sampling is configured in code as ParentBased with a root trace ratio of 1.0, following the parent Span's sampling decision. In production, you can change the ratio in the example to 0.1, sampling approximately 10% of root traces. This example sets the sampler explicitly and does not rely on OTEL_TRACES_SAMPLER or OTEL_TRACES_SAMPLER_ARG.

The example explicitly creates the trace export pipeline; OTEL_TRACES_EXPORTER does not select or close it. No Metric or Log Provider is created, so setting OTEL_METRICS_EXPORTER or OTEL_LOGS_EXPORTER will not enable the corresponding signals. For logs, you can separately use DataKit Log File Collection.

Context Propagation and Application Integration¶

This example registers the W3C TraceContext propagator but does not automatically intercept network requests. For HTTP/RPC integration, use an Extractor on the server side to extract the upstream context and use it as the parent of the new Span, and use an Injector on the client side to inject traceparent and tracestate. The in_span in the synchronous example should not be used directly across .await; asynchronous tasks should propagate context with FutureExt::with_context or similar. Applications using tracing also need to configure a compatible version of the tracing-opentelemetry Layer.

Verification and Troubleshooting¶

1. After running the example, query traces by order-service in TrueWatch APM and confirm that checkout and its child Span db.lookup are present.
2. If no data appears, check whether the collector is enabled, whether the application environment variables took effect, whether the HTTP path includes /otel/v1/traces, and the export errors from DataKit and the application.
3. Confirm that Spans have ended and the Provider completes flushing before the process exits. Force-quitting or using a root trace ratio of 0 will prevent the expected data from appearing; network connectivity does not mean export succeeded.

References¶

• OpenTelemetry Rust official documentation
• SDK and example source code
• Exporter