OpenTelemetry Go (LoongSuite)¶

LoongSuite Go is an OpenTelemetry-based compile-time automatic instrumentation tool for Go. It does not require modifying business code – simply replace go build with otel go build to inject the OpenTelemetry SDK and instrumentation logic for supported frameworks and components during compilation.

This document uses the DataKit OpenTelemetry collector to receive traces and metrics reported by LoongSuite via OTLP and forward them to TrueWatch:

Go source code -- otel go build --> Instrumented Go binary -- OTLP --> DataKit --> TrueWatch

!!! note

LoongSuite's "zero-code" means no changes to business code, but it does not mean no rebuild is required. Instrumentation happens at compile time; an already-built Go binary cannot be retrofitted with LoongSuite. You must recompile with `otel go build` and deploy the new binary.

Prerequisites¶

• DataKit is installed and connected to the target TrueWatch workspace.
• Network reachability from the Go application to DataKit: OTLP/HTTP uses DataKit HTTP port 9529, OTLP/gRPC defaults to 4317.
• The application can be compiled normally with go build.
• Go version, OS, and architecture meet LoongSuite's compatibility requirements.
• The frameworks or components used by the application are in LoongSuite's supported libraries list.

This document uses a Linux AMD64 host as an example and does not cover Kubernetes deployment.

1. Enable the OpenTelemetry Collector¶

Navigate to the conf.d/opentelemetry directory under the DataKit installation directory. If the collector configuration does not exist, copy the sample file:

cd /usr/local/datakit/conf.d/opentelemetry
sudo cp opentelemetry.conf.sample opentelemetry.conf

Ensure that opentelemetry.conf includes at least the following receiver configuration:

[[inputs.opentelemetry]]
  # If you want to keep custom attributes as tags, add them to the allowlist.
  # Dots in attribute names are converted to underscores, e.g. team.name -> team_name.
  customer_tags = ["team", "project"]

  [inputs.opentelemetry.http]
    http_status_ok = 200
    trace_api = "/otel/v1/traces"
    metric_api = "/otel/v1/metrics"

  [inputs.opentelemetry.grpc]
    addr = "127.0.0.1:4317"
    max_payload = 16777216

The configuration above enables the following receiver endpoints:

If the application is not on the same host as DataKit, adjust the DataKit HTTP listener address, firewall, or other network access controls accordingly. When using OTLP/gRPC, change addr to a listener address accessible by the application, e.g., 0.0.0.0:4317. Do not expose OTLP receiver ports directly to the public internet.

Restart DataKit to apply the configuration:

sudo datakit service restart

Verify that the DataKit HTTP service is reachable:

curl http://127.0.0.1:9529/v1/ping

2. Instrument the Application with OpenTelemetry¶

Install LoongSuite¶

Download the Linux AMD64 executable from the official LoongSuite GitHub Release:

sudo curl -fL \
  https://github.com/alibaba/loongsuite-go/releases/latest/download/otel-linux-amd64 \
  -o /usr/local/bin/otel
sudo chmod +x /usr/local/bin/otel

For ARM64 hosts, replace the filename with otel-linux-arm64. For other OS and architectures, select the appropriate file from LoongSuite Releases.

Confirm the tool works:

otel version
go version

For production environments, it is recommended to pin the LoongSuite version by using a Release download URL with an explicit version number, and check compatibility, Release Notes, and perform compilation and trace regression testing before upgrading.

Pre-build Check¶

First, confirm that the project compiles normally with the original command:

go build ./...

If the project already directly depends on the OpenTelemetry Go API, SDK, or Contrib instrumentation libraries, check whether these dependencies are compatible with the current LoongSuite version:

go list -m all | grep 'go.opentelemetry.io'

LoongSuite injects SDK initialization logic and instruments OpenTelemetry itself. If the project already has incompatible OpenTelemetry dependencies or duplicate SDK initialization logic, compilation failures, duplicate spans, or context breaks may occur. In such cases, unify the dependency versions according to the official compatibility table. If the application needs to control the SDK itself, consider using the OpenTelemetry Go SDK instead.

Build with LoongSuite¶

Navigate to the Go project directory and prepend otel to the original build command:

otel go build -o ./bin/order-service ./cmd/order-service

Other common build patterns retain the original go build arguments, for example:

otel go build
otel go build -trimpath -ldflags="-s -w" -o ./bin/order-service ./cmd/order-service

LoongSuite compilation adds pre-processing, instrumentation, and dependency handling phases. The first build is usually significantly slower than native go build. You can configure a reusable Go build cache to speed up subsequent builds:

otel set -gocache=/var/tmp/loongsuite-go-cache

You can also specify the cache via environment variables for CI or single builds:

export OTELTOOL_GO_CACHE="/var/tmp/loongsuite-go-cache"
otel go build -o ./bin/order-service ./cmd/order-service

!!! warning

Subsequent deployment processes must use the binary produced by `otel go build`. If you overwrite the artifact with a regular `go build`, the runtime will not include LoongSuite's automatic instrumentation.

Configure OTLP/HTTP and Start¶

The following example reports traces and metrics to the local DataKit via OTLP/HTTP + Protobuf. After setting the common endpoint to http://127.0.0.1:9529/otel, the exporter appends /v1/traces and /v1/metrics respectively, which correspond to DataKit's /otel/v1/* receiver paths.

export OTEL_SERVICE_NAME="order-service"
export OTEL_RESOURCE_ATTRIBUTES="deployment.environment.name=prod,service.version=1.0.0,team=backend"

export OTEL_TRACES_EXPORTER="otlp"
export OTEL_METRICS_EXPORTER="otlp"
export OTEL_EXPORTER_OTLP_PROTOCOL="http/protobuf"
export OTEL_EXPORTER_OTLP_ENDPOINT="http://127.0.0.1:9529/otel"
export OTEL_EXPORTER_OTLP_INSECURE="true"

# 1.0 means full sampling; only recommended during integration verification.
export OTEL_TRACE_SAMPLER="1.0"

./bin/order-service

The runtime parameters must be configured in the actual runtime environment of the instrumented binary, not just on the build host. After startup, request an endpoint provided by a supported framework, triggering database, HTTP client, or message queue calls to produce verifiable spans and metrics.

Use OTLP/gRPC¶

To switch to OTLP/gRPC, replace the protocol and endpoint. The gRPC address must not include HTTP paths like /v1/traces:

export OTEL_EXPORTER_OTLP_PROTOCOL="grpc"
export OTEL_EXPORTER_OTLP_ENDPOINT="http://127.0.0.1:4317"
export OTEL_EXPORTER_OTLP_INSECURE="true"

3. Data Reporting Parameters¶

LoongSuite injects OpenTelemetry SDK initialization logic during compilation. The instrumented application can adjust the exporter, endpoint, sampling, and resource attributes via environment variables at runtime.

Resource and Exporter Parameters¶

When using the generic HTTP endpoint, set it to http://<DataKit-IP>:9529/otel. If using signal-specific endpoints, you must include the full path containing /otel/v1/traces or /otel/v1/metrics. DataKit's OTLP/HTTP collector only supports Protobuf; do not configure http/json.

Sampling and Metric Parameters¶

!!! warning

LoongSuite currently uses the environment variable `OTEL_TRACE_SAMPLER`, which differs from the common OpenTelemetry SDK names `OTEL_TRACES_SAMPLER` and `OTEL_TRACES_SAMPLER_ARG`. Refer to the [SDK configuration documentation](https://github.com/alibaba/loongsuite-go/blob/main/docs/user/sdk-config.md){:target="_blank"} for the installed LoongSuite version.

In production, adjust the sampling rate based on traffic, data budget, and troubleshooting needs. Upstream and downstream services must maintain compatible W3C Trace Context propagation to avoid broken spans across service calls.

LoongSuite Build Parameters¶

The following variables only control the LoongSuite build tool, not the data reporting of the instrumented binary:

For example, to temporarily output detailed build logs:

export OTELTOOL_DEBUG="true"
export OTELTOOL_VERBOSE="true"
otel go build -o ./bin/order-service ./cmd/order-service

After troubleshooting, disable Debug and Verbose to reduce log volume.

Field Mapping¶

OTLP span attributes reported by LoongSuite are converted to trace fields by the DataKit OpenTelemetry collector. Common mappings are as follows:

To promote other attributes reported by LoongSuite to tags, configure customer_tags in the DataKit opentelemetry.conf. customer_tags supports regular expressions; matched attribute names have dots converted to underscores:

[[inputs.opentelemetry]]
  customer_tags = [
    "reg:^db\\.query\\.parameter\\.",
    "reg:^kratos\\.service\\.meta\\.",
    "reg:^gen_ai\\.other_input\\.",
    "reg:^gen_ai\\.other_output\\.",
  ]

Do not promote high-cardinality values such as user IDs or order numbers in bulk to tags, and do not report sensitive information such as passwords, tokens, or full database connection strings.

Verify Integration¶

1. Compile and start the new binary using otel go build.
2. Request an endpoint handled by a supported web framework, triggering database, HTTP client, or message queue calls.
3. When using OTLP/HTTP, check the receiver logs on the DataKit host:

sudo tail -f /usr/local/datakit/log/gin.log | grep '/otel/v1/'

If you see POST requests to /otel/v1/traces or /otel/v1/metrics with a 200 response code, DataKit has received the data. Then go to Application Performance Monitoring > Traces in TrueWatch and query by service:order-service. For metrics, wait at least one export cycle before querying.

If no data is received, check the following in order:

• Is the deployed binary the one produced by otel go build?
• Are OTEL_SERVICE_NAME, exporter, protocol, and endpoint configured in the actual runtime process?
• Is the framework and version used by the application in the LoongSuite supported libraries list?
• Is the LoongSuite version compatible with the project's existing OpenTelemetry dependencies?
• Is the DataKit OpenTelemetry collector enabled and restarted?
• Is the network and port from the application to DataKit reachable?

If go build succeeds but otel go build fails, temporarily enable OTELTOOL_DEBUG=true and OTELTOOL_VERBOSE=true to identify the specific failing instrumentation rule. If necessary, temporarily disable the problematic rule with otel set -disable=<rule-name> and report the issue to LoongSuite Issues.

References¶

• LoongSuite Go
• LoongSuite Supported Libraries
• LoongSuite SDK Configuration
• LoongSuite Build Tool Configuration
• OpenTelemetry Zero-Code Instrumentation for Go
• DataKit OpenTelemetry Collector