Skip to content

Browser Dialtesting

Version-2.1.0


Browser dialtesting is a BROWSER task type under the inputs.dialtesting collector. It simulates page access through Lightpanda or Chrome, including opening pages, clicking elements, entering text, waiting for selectors, and asserting titles or text. It reports page performance, step details, failure reasons, and optional failure screenshots.

For basic dialtesting node configuration, see Network Dialtesting. This page only describes browser-specific configuration, deployment, and troubleshooting.

Enable Browser Dialtesting

Browser dialtesting is enabled by default. To configure it explicitly, set it in dialtesting.conf:

[[inputs.dialtesting]]
  # Other dialtesting node settings are omitted.

  [inputs.dialtesting.browser]
    enabled = true

Browser dialtesting currently supports Linux dialtesting nodes only. On non-Linux platforms, DataKit service mode does not run BROWSER tasks even when enabled = true is configured, except when using local debug verification mode.

DataKit defaults to Lightpanda for BROWSER tasks. Explicitly select the optional chrome engine to run official Chrome Headless Shell. Task-level advance_options.engine is overwritten by the node configuration. The Chrome option uses the unmodified official binary with runtime flags; it does not build or use custom Chromium.

Chrome Headless Shell

The datakit:<version> image bundles Lightpanda and official Chrome Headless Shell, defaulting to Lightpanda. Host deployments need a separately installed browser. Select chrome and point engine_path to the executable:

[inputs.dialtesting.browser]
  engine = "chrome"
  engine_path = "/usr/local/bin/chrome-headless-shell"
  load_images = false
  load_fonts = false
  allow_no_sandbox = true
  max_concurrency = 1

load_images and load_fonts both default to false. Set either to true to restore image or web-font loading. These settings only affect Chrome/Chrome Headless Shell, not Lightpanda. The corresponding environment variables are ENV_INPUT_DIALTESTING_BROWSER_LOAD_IMAGES and ENV_INPUT_DIALTESTING_BROWSER_LOAD_FONTS.

allow_no_sandbox is a node-level dialtesting input setting, defaults to true, and maps to ENV_INPUT_DIALTESTING_BROWSER_ALLOW_NO_SANDBOX. It does not affect Lightpanda. Non-root users still try Chrome's sandbox first; only an explicit sandbox-unavailable startup error permits one retry without it, with a warning. Root users start without the sandbox when compatibility mode is allowed. To require the browser sandbox, set allow_no_sandbox = false or set the environment variable to false, use a non-root account, and configure a working sandbox. Root startup or an unavailable sandbox then fails explicitly. Task scripts cannot override the node setting. The default prioritizes startup compatibility; disabling the sandbox removes that layer of browser isolation.

The DK image sets executable-path environment variables, so engine_path can be omitted when switching engines. Set ENV_INPUT_DIALTESTING_BROWSER_ENGINE to lightpanda or chrome; the default remains lightpanda. The Chrome entry point adds low-memory runtime flags to the official Shell. Set engine_path to /usr/local/bin/chrome-headless-shell-standard to use the same official binary without that flag profile.

Chrome executable lookup uses engine_path, then CHROME_EXECUTABLE_PATH, then chrome-headless-shell on PATH. An unusable configured path fails explicitly instead of falling back to another browser. Automatic lookup does not select desktop Chrome, Chromium, or an experimental headless_shell.

Disabled resources do not simulate successful loading. Enable the required resources for image-load handlers, CAPTCHAs, QR codes, font-loading dependencies, screenshots, or page-performance measurements. Font fallback can change layout. These node-level settings apply to the node's Chrome browser tasks.

Chrome enforces the private-network/CIDR policies from disable_internal_network_task through a per-task controlled proxy; see Private Network and Proxy. It still does not support browser.ca_cert_file / browser.ca_cert_dir; configuring them fails the task explicitly. Chrome uses the system certificate trust store and retains site isolation; the node's allow_no_sandbox setting controls sandbox behavior. Each task currently starts an independent browser; browser processes and sessions are not shared across tenants.

DataKit resolves Lightpanda in the following order:

  1. [inputs.dialtesting.browser].engine_path
  2. LIGHTPANDA_EXECUTABLE_PATH
  3. lightpanda from PATH

max_concurrency limits the total browser tasks running at the same time in this input, defaulting to 8 when omitted. Tasks above the limit wait, which can increase scheduling delay. 0 means no limit; an existing explicit 0 remains unlimited after upgrading, so change it to 8 to apply the limit. On resource-limited nodes, 1 is recommended.

Kubernetes Deployment

Select the browser through the Deployment container environment, for example to use Chrome Headless Shell:

env:
  - name: ENV_INPUT_DIALTESTING_BROWSER_ENGINE
    value: "chrome"
  - name: ENV_INPUT_DIALTESTING_BROWSER_ENGINE_PATH
    value: "/usr/local/bin/chrome-headless-shell"

With the DK image, omit ENV_INPUT_DIALTESTING_BROWSER_ENGINE_PATH to use its default executable path. To switch back, set the engine to lightpanda; if an explicit path is configured, also change it to /usr/local/bin/lightpanda. The configuration applies to browser tasks handled by this input and overrides task advance_options.engine. Recreate the Pod after updating the Deployment so the new environment takes effect.

For Kubernetes, use the regular DK image:

pubrepo.truewatch.com/truewatch/datakit:<version>

Production and testing releases use the regular DK image tag for both engines. For example, when the release version is 2.18.0:

Purpose Example image tag
Production image datakit:2.18.0
Testing image datakit:2.18.0-testing_testing-3308

Lightpanda remains the default. Explicitly set engine = "chrome" to select official Headless Shell.

The Ubuntu 24.04 image supports Linux amd64 and arm64. It includes official Lightpanda 1.0.2, official Chrome Headless Shell 157.0.8103.38, and Chinese/Latin fonts. Browser files use a reusable layer, so DK updates can reuse cached browser files. Database clients, Python, Jolokia, and external collectors under externals remain included.

The image retains the regular DK root account by default. Root starts Chrome without its sandbox when allow_no_sandbox = true. To require the sandbox, run as a non-root account, such as the included UID/GID 10001:10001, set a writable HOME, and allow this user access to mounted configuration, cache, and log directories. Set allow_no_sandbox = false and permit the required user namespaces in the container runtime. Raw-socket ICMP/NETPATH features also need appropriate network permissions.

When retaining Chrome's sandbox, Docker's default seccomp policy may block browser user namespaces. If required by the deployment, download the official Playwright v1.64.1 profile and reference it with docker run --security-opt seccomp=./seccomp_profile.json .... For Kubernetes, a node administrator can provide an equivalent runtime policy permitting browser user namespaces. The profile only supplies the system calls needed for the browser sandbox; retain separate tenant containers and egress policies.

To use a custom Lightpanda binary, mount the executable into the container and configure engine_path:

[[inputs.dialtesting]]
  [inputs.dialtesting.browser]
    enabled = true
    engine = "lightpanda"
    engine_path = "/opt/datakit-browser/bin/lightpanda"
    max_concurrency = 8

Host Deployment

For host deployment, select an engine and install that browser. Installing both engines is optional. The DataKit host installer does not currently install Chrome Headless Shell automatically. Install a fixed browser version once and reuse it across normal DK upgrades.

Choice Host requirements
Lightpanda The Lightpanda executable and its required system libraries; see the installation below
Chrome Official Chrome Headless Shell, with the corresponding system libraries, fonts, and sandbox configuration

The following Linux examples use a fixed official Headless Shell version. DataKit starts browsers per task; a separate persistent browser service is not required.

Install Lightpanda

The DK image uses the official Lightpanda 1.0.2 release with pinned URLs and SHA-256 checksums, using the Ubuntu 24.04 runtime libraries. For host installation, its amd64 binary requires glibc 2.35, while its arm64 binary references glibc 2.38 symbols and cannot run directly on Ubuntu 22.04. To install official 1.0.2 on x86_64 Linux:

curl -fL -o lightpanda \
  https://github.com/lightpanda-io/browser/releases/download/1.0.2/lightpanda-x86_64-linux
echo "ae7d01d261efb9e480e18ce3424533332883863bcf70b12a4321b5733f37e19e  lightpanda" | sha256sum -c -
sudo install -m 0755 lightpanda /usr/local/bin/lightpanda
rm lightpanda

For arm64/aarch64 Linux, use:

curl -fL -o lightpanda \
  https://github.com/lightpanda-io/browser/releases/download/1.0.2/lightpanda-aarch64-linux
echo "ac546dfc43c533430da92b75ab36f2a29da7d6324d5d38baf606ef2e0d849345  lightpanda" | sha256sum -c -
sudo install -m 0755 lightpanda /usr/local/bin/lightpanda
rm lightpanda

Verify the installation:

lightpanda version
lightpanda serve --help

Install Headless Shell

This example pins official Chrome Headless Shell 157.0.8103.38 for Linux amd64 and arm64. Update the version, download URL, and SHA-256 together. Downloads come from Chrome for Testing. When Google Storage is unavailable or slow, the example falls back to Microsoft's Playwright mirror and checks the same hash. Use another download site only after it hosts the matching artifact and its hash has been verified.

Prepare curl, unzip, and CA certificates, for example on Ubuntu:

sudo apt-get update
sudo apt-get install -y --no-install-recommends ca-certificates curl unzip

Download, verify, and install into a separate version directory:

(
  set -eu
  version=157.0.8103.38
  case "$(uname -m)" in
    x86_64|amd64)
      platform=linux64
      sha256=deafffa74660b34853a4119c361fd427ac8c9fcb62db51ddb8a797cd83a4ba2d
      ;;
    aarch64|arm64)
      platform=linux-arm64
      sha256=b9b910a57b2a15eb3e55000a2339f93a973934a413ebfc6e585a50ec6c42fd6c
      ;;
    *) echo "Unsupported architecture: $(uname -m)" >&2; exit 1 ;;
  esac
  install_dir="/opt/datakit-browser/chrome-headless-shell-$version"
  if [ -e "$install_dir" ]; then
    echo "Target exists: $install_dir; use the installed version or a new version directory" >&2
    exit 1
  fi
  download_dir=$(mktemp -d)
  trap 'rm -rf -- "$download_dir"' EXIT
  archive="$download_dir/chrome.zip"
  download_path="$version/$platform/chrome-headless-shell-$platform.zip"
  if ! curl -fL --retry 1 --connect-timeout 10 --max-time 180 --speed-limit 65536 --speed-time 20 \
      "https://storage.googleapis.com/chrome-for-testing-public/$download_path" -o "$archive"; then
    curl -fL --retry 3 --connect-timeout 10 --max-time 180 \
      "https://cdn.playwright.dev/builds/cft/$download_path" -o "$archive"
  fi
  echo "$sha256  $archive" | sha256sum -c -
  unzip -q "$archive" -d "$download_dir"
  sudo install -d -m 0755 /opt/datakit-browser
  sudo mv -T "$download_dir/chrome-headless-shell-$platform" "$install_dir"
  sudo chown -R root:root "$install_dir"
  sudo chmod 0755 "$install_dir" "$install_dir/chrome-headless-shell"
)

Headless Shell Requirements

Headless Shell retains Chromium page execution and rendering. Its shared-library dependencies include GLib, NSS, X11/GBM, ATK, and ALSA. These libraries have transitive dependencies, so the package count can be substantial. The package manager resolves them together and reuses installed packages. A desktop environment or running X Server is not required. Dependencies vary by browser version, distribution, and architecture; a general Chrome list is not a mandatory package list for every host.

Check the current host for missing shared libraries first:

ldd /opt/datakit-browser/chrome-headless-shell-157.0.8103.38/chrome-headless-shell

An entry marked not found identifies a missing runtime library. The following Ubuntu 24.04 example is not a universal minimum for other distributions:

sudo apt-get install -y --no-install-recommends \
  ca-certificates libstdc++6 libatomic1 libsystemd0 \
  libasound2t64 libatk-bridge2.0-0t64 libatk1.0-0t64 libatspi2.0-0t64 \
  libcairo2 libcups2t64 libdbus-1-3 libdrm2 libgbm1 libglib2.0-0t64 \
  libnspr4 libnss3 libpango-1.0-0 libx11-6 libxcb1 libxcomposite1 \
  libxdamage1 libxext6 libxfixes3 libxkbcommon0 libxrandr2

Package names differ on Ubuntu 22.04, Debian, and other distributions, especially names ending in t64; do not copy this command unchanged to those systems. Use the Chrome Linux dependency guide with the ldd output. A GLIBC_x.y not found error requires a compatible browser build or runtime environment; do not replace the system glibc directly. After checking ldd, verify browser startup and an actual dialtesting task.

Install fonts according to page requirements. For example, Latin and Chinese screenshots on Ubuntu can use:

sudo apt-get install -y --no-install-recommends fonts-liberation fonts-noto-cjk
# Install only when pages need Emoji:
# sudo apt-get install -y --no-install-recommends fonts-noto-color-emoji

load_fonts = false blocks web-font downloads. System fallback fonts still affect layout and screenshots. Missing Chinese fonts can produce boxes, and different fonts can change layout, so include font selection in the dialtesting node environment.

Allow disk space for the browser, libraries, and fonts. Use the downloaded artifact and package manager estimates for your host. Disk usage does not predict runtime memory.

Check the version as the account that actually runs DataKit, for example if the service account is datakit:

sudo -u datakit /opt/datakit-browser/chrome-headless-shell-157.0.8103.38/chrome-headless-shell --version

--version only checks the binary and basic runtime libraries; it does not validate the sandbox or a full page flow. The browser inherits the DataKit service account; running sudo -u datakit once does not change that account. Prefer a non-root dialtesting DK instance, with writable configuration, cache, and log directories for its account. To retain Chrome's sandbox, set allow_no_sandbox = false and permit its required user namespaces. Ubuntu 24.04 and other systems may also need an AppArmor rule for the browser installation path; follow the Chromium user-namespace guidance. The Docker seccomp file discussed above is for containers and does not replace the host's AppArmor configuration. Only when compatibility mode is allowed does DK retry an unavailable sandbox with --no-sandbox; other startup errors do not trigger this retry.

Configure DataKit

Copy the dialtesting input configuration:

cd /usr/local/datakit/conf.d/samples
sudo cp dialtesting.conf.sample ../dialtesting.conf

Edit /usr/local/datakit/conf.d/dialtesting.conf, confirm browser dialtesting remains enabled, and set the browser engine and path explicitly:

[[inputs.dialtesting]]
  server = "https://dflux-dial.truewatch.com"
  region_id = "<your-private-node-id>"
  ak = "<your-ak>"
  sk = "<your-sk>"
  pull_interval = "1m"
  time_out = "30s"

  [inputs.dialtesting.browser]
    enabled = true
    engine = "lightpanda"
    engine_path = "/usr/local/bin/lightpanda"
    max_concurrency = 8

  [inputs.dialtesting.tags]
    region = "<your-region>"

For Chrome, replace the browser section of the same dialtesting input with the following. Do not add a second inputs.dialtesting instance:

[inputs.dialtesting.browser]
  engine = "chrome"
  engine_path = "/opt/datakit-browser/chrome-headless-shell-157.0.8103.38/chrome-headless-shell"
  load_images = false
  load_fonts = false
  max_concurrency = 1

Enable load_images and load_fonts as required for images, web fonts, or complete screenshots. The Chrome network policies and custom-CA limitations described above also apply to host deployments.

Alternatively, set the executable-path environment variable for the selected engine:

export LIGHTPANDA_EXECUTABLE_PATH=/usr/local/bin/lightpanda
# For the chrome engine:
# export CHROME_EXECUTABLE_PATH=/opt/datakit-browser/chrome-headless-shell-157.0.8103.38/chrome-headless-shell

An executable-path environment variable does not select the engine; still configure engine = "chrome" or engine = "lightpanda". If DataKit runs as a systemd service, exporting a variable in the current shell usually does not pass it to the service process. Setting engine_path in dialtesting.conf is recommended. If using environment variables, write them into the service environment configuration and restart the service.

Restart DataKit after updating the configuration:

sudo datakit service restart

Custom CA Certificates

For sites whose certificates are issued by an enterprise or private CA, import the trusted CA used by Lightpanda on the dialtesting node:

[inputs.dialtesting.browser]
  ca_cert_file = "/etc/datakit/certs/internal-ca.pem"
  # ca_cert_dir = "/etc/datakit/certs"

Use PEM certificates. ca_cert_dir loads certificate files from a directory, and both settings can be used together. Paths must be absolute paths on the dialtesting node, and certificate files must not contain private keys.

The equivalent environment variables are ENV_INPUT_DIALTESTING_BROWSER_CA_CERT_FILE and ENV_INPUT_DIALTESTING_BROWSER_CA_CERT_DIR. In Kubernetes, mount the CA certificate into the DataKit container through a ConfigMap or Secret and configure its in-container path.

The imported CA does not disable TLS verification: the engine still verifies the certificate chain, hostname, and validity period. This is a node-level trust setting and cannot be overridden by an individual BROWSER task. Lightpanda itself replaces the active trust store when custom CA arguments are present, so DataKit passes a detected system CA directory and the configured custom CA file or directory as separate arguments. The system and custom certificate files remain in their original directories while both are loaded into the same in-memory trust store.

Private Network and Proxy

DataKit blocks internal network targets by default. On a private dialtesting node that must access loopback, RFC1918, or link-local addresses, disable this restriction in the dialtesting input:

[[inputs.dialtesting]]
  disable_internal_network_task = false

DataKit also passes this setting to Lightpanda. With the default disable_internal_network_task = true and no custom CIDR list, Lightpanda starts with --block-private-networks. When disabled_internal_network_cidr_list is configured, DataKit passes those exact ranges through --block-cidrs instead of blocking every private range. When disable_internal_network_task is false, Lightpanda allows private-network requests. No engine-specific environment variable is required.

Chrome uses the same node settings. When restrictions are enabled, DataKit starts an in-process controlled proxy on a random loopback port for each task. Completion, startup failure, or cancellation closes the proxy and its connections. No separate proxy installation or exposed container port is required. HTTP, HTTPS, and WebSocket requests use the proxy, including page resources and requests following redirects. Before each new destination connection, the proxy checks every resolved IPv4/IPv6 address and rejects the destination if any is denied. It connects directly to a checked IP without a second lookup. HTTPS uses CONNECT tunnels and preserves browser-to-origin TLS and certificate validation.

DataKit manages the built-in controlled proxy automatically; proxy_url is not required. Without that setting, the controlled proxy checks addresses and connects directly to the origin, while private-network/CIDR restrictions remain active. Set proxy_url only to add an upstream proxy. The upstream protocol, authentication, and CONNECT requirements below apply only to that additional proxy.

Linux host and container deployments use the same mechanism. It does not depend on Kubernetes or NetworkPolicy and does not automatically modify host or cluster firewall rules. Deployment operators manage platform network policies.

The default ranges include private, loopback, link-local, CGNAT, and 198.18.0.0/15 benchmark addresses. A custom disabled_internal_network_cidr_list replaces these default ranges, matching Lightpanda's configuration semantics. A public-looking hostname is still blocked if the node resolves it to a denied address. An invalid CIDR explicitly fails the Chrome task.

Controlled mode disables Chrome's implicit loopback proxy bypass, QUIC, and non-proxied WebRTC UDP. HTTP/3 and direct UDP media paths are not measured in this mode. Browser flags and the application proxy do not replace container/Pod egress policies. Use separate container boundaries for tenants and restrict allowed outbound destinations and protocols to address other network paths or browser vulnerabilities. Loopback ports in a shared network namespace are not a tenant isolation boundary.

Sites that also support HTTP/2 or HTTP/1.1 can still be tested through those fallback protocols; the results do not measure HTTP/3 connection performance. Sites requiring HTTP/3 are not supported in controlled mode. Native HTTP dialtesting's direct HTTP/3 support is separate from this browser proxy and currently does not support proxy configuration either.

Lightpanda 1.0.2 supports the following default HTTP proxy setting:

[inputs.dialtesting.browser]
  proxy_url = "http://proxy.example.com:8080"
  # proxy_url = "http://user:password@proxy.example.com:8080"

The equivalent environment variable is ENV_INPUT_DIALTESTING_BROWSER_PROXY_URL. Proxy precedence is: task advance_options.proxy_url > proxy_url in browser_config > node-level browser.proxy_url. When using Lightpanda with a proxy that intercepts HTTPS traffic, import its CA through ca_cert_file or ca_cert_dir as well.

Chrome uses the same proxy_url and priority, supporting HTTP/HTTPS upstream proxies with Basic credentials in the URL. DataKit connects through its per-task controlled proxy; credentials are neither passed in Chrome command-line arguments nor forwarded to the origin. The upstream must support CONNECT to resolved destination IP:port addresses, including ports used for HTTP sites. SOCKS, PAC, and authentication such as NTLM are unsupported, and failures never fall back to direct connections. The dialtesting node resolves origin names, so environments depending on upstream-only DNS must configure node resolution first.

The node administrator's [inputs.dialtesting.browser].proxy_url, including its node environment variable, is a trusted upstream whose endpoint may use an internal address. The final origin remains subject to the private-network/CIDR policy. This trust applies only when the node proxy is selected. An override from task advance_options.proxy_url or browser_config causes the upstream's resolved IPs to be checked against the node CIDR policy as well. Even an override with the same URL does not inherit node trust. Blank proxy values are treated as unset and fall through to the next priority.

HTTPS upstream proxies use the node system CA store and always validate certificates, regardless of task ignore_https_errors. Chrome does not support the ca_cert_file / ca_cert_dir settings above; manage enterprise CA trust through the deployment environment's system trust store.

Verify with a Local Task

If no BROWSER task is available from the console yet, use a local JSON task to verify the browser execution path first. browser_config is a YAML string. It is easier to write the browser script as YAML first, then put it into the JSON task.

Browser script example:

name: browser-homepage
target: https://example.com
timeout_ms: 60000
viewport:
  width: 1280
  height: 720
steps:
  - name: open page
    action: goto
    url: https://example.com
  - name: assert title
    action: assert_title
    contains: Example

Create /tmp/dialtesting-browser-task.json. When writing JSON, put the YAML above into browser_config as a string and represent line breaks with \n:

{
  "BROWSER": [
    {
      "name": "browser-homepage",
      "url": "https://example.com",
      "status": "OK",
      "frequency": "1m",
      "post_url": "https://openway.truewatch.com?token=<your-token>",
      "browser_config": "name: browser-homepage\ntarget: https://example.com\ntimeout_ms: 60000\nviewport:\n  width: 1280\n  height: 720\nsteps:\n  - name: open page\n    action: goto\n    url: https://example.com\n  - name: assert title\n    action: assert_title\n    contains: Example\n"
    }
  ]
}

Temporarily set server in dialtesting.conf to the local file URL and keep browser dialtesting enabled:

[[inputs.dialtesting]]
  server = "file:///tmp/dialtesting-browser-task.json"
  pull_interval = "10s"
  time_out = "30s"

  [inputs.dialtesting.browser]
    enabled = true
    engine = "lightpanda"
    engine_path = "/usr/local/bin/lightpanda"
    max_concurrency = 8

After verification, restore server, region_id, ak, sk, and other settings to the real dialtesting node configuration.

Then verify with debug mode or by restarting DataKit:

datakit debug --input-conf /usr/local/datakit/conf.d/dialtesting.conf

If DataKit runs as a service:

sudo datakit service restart

After 1~2 pull intervals, check metrics:

curl -s http://127.0.0.1:9529/metrics | grep datakit_dialtesting

Normally, datakit_dialtesting_task_number{protocol="BROWSER"} is greater than 0, and datakit_dialtesting_worker_send_points_number{protocol="BROWSER",status="ok"} keeps increasing.

BROWSER Task Example

In custom dialtesting tasks, a BROWSER task uses browser_config to define the browser script. browser_config is a YAML string that describes page navigation, interactions, and assertions.

Common browser_config fields:

Field Type Required Description
name string N Script name
target string N Default target URL, used when a goto step does not configure a URL
timeout_ms int N Script timeout in milliseconds
viewport.width int N Browser viewport width
viewport.height int N Browser viewport height
tags object N Custom tags
steps array Y Browser execution steps

steps can use actions and assertions such as goto, click, fill, wait_for_selector, wait_for_url, assert_title, assert_url, and assert_text.

wait_for_url accepts contains, equals, or text and polls until the URL matches or the step/script timeout expires. assert_title, assert_url, and assert_text always poll as well. A configured step timeout_ms takes precedence; otherwise the script timeout applies. DataKit uses a 60-second default step timeout. For example:

- name: wait for dashboard redirect
  action: wait_for_url
  contains: https://console.example.com/dashboard
  timeout_ms: 15000
- name: assert dashboard title
  action: assert_title
  contains: Dashboard
  timeout_ms: 5000

In the full task JSON, browser_config is inside the BROWSER task object:

{
  "BROWSER": [
    {
      "name": "browser-homepage",
      "url": "https://example.com",
      "status": "OK",
      "frequency": "1m",
      "post_url": "https://openway.truewatch.com?token=<your-token>",
      "browser_config": "<browser_config YAML string>",
      "advance_options": {
        "screenshot_on_failure": true
      }
    }
  ]
}

Screenshot Support

Set advance_options.screenshot_on_failure to true to capture a PNG after a browser step fails. DataKit uploads the image and records its metadata in steps[].screenshot; upload failures are reported in screenshot_upload_error.

Lightpanda screenshots are text-oriented semantic renderings rather than pixel-perfect browser images. They are useful for checking the page content and DOM state at the point of failure, but do not reproduce all CSS, images, canvas content, or other Chrome rendering details. A screenshot is unavailable when Lightpanda cannot start, the browser process exits, or the browser session has already reached its overall timeout.

Correlating RUM Data

Use Chrome/Chrome Headless Shell when testing with Web RUM SDK 3.3.22. Lightpanda 1.0.2 fixes the older version's time origin, but the SDK still needs to convert fractional millisecond timestamps to the integer milliseconds required by its intake format. The context injection below does not change the SDK reporting format.

When the target page already uses a Web RUM SDK that supports browser synthetic correlation, the runner supplies the following context before page scripts execute:

Global variable Value SDK event field
window._DATAFLUX_SYNTHETICS_BROWSER true session_type = synthetics
window._GUANCE_SYNTHETICS_PUBLIC_ID Platform task external_id synthetics_test_id
window._GUANCE_SYNTHETICS_RESULT_ID Current attempt's browser_run_id synthetics_result_id

The context is installed in new documents after navigation, reload, and navigation across origins in the controlled page. Each attempt uses an isolated browser environment and result ID. The SDK can correlate View, Action, Resource, Error, and LongTask events. The runner does not install or initialize the SDK, change sampling, or enable session replay. Local scripts without a platform task ID do not set these variables.

The top-level browser_run_id continues to identify the final attempt. With retries enabled, retry_records[].browser_run_id preserves each attempt's result ID so RUM queries can locate both failed and successful attempts using the task ID. Older records may lack this field; do not infer precise correlation from URL and time. An attempt can produce multiple sessions or views, so queries must retain all matching objects.

This covers documents in the page controlled by the runner. Independent popups and new tabs are not guaranteed to receive the context, and the runner does not automatically switch to them. Data availability depends on SDK compatibility, sampling, reporting, ingestion, and retention. Ending an execution does not guarantee that the SDK has flushed its events. This establishes collection context only; RUM query authorization and detail navigation are implemented separately. Queries must still enforce workspace, application, and data access permissions.

Troubleshooting

Use DataKit metrics on the dialtesting node to check task and reporting status:

curl -s http://127.0.0.1:9529/metrics | grep datakit_dialtesting

Check these metrics first:

datakit_dialtesting_task_number
datakit_dialtesting_worker_send_points_number
datakit_dialtesting_dataway_send_failed_number
datakit_dialtesting_worker_cached_points_number
datakit_dialtesting_worker_dropped_points_number

Check browser engine availability with:

echo $LIGHTPANDA_EXECUTABLE_PATH
$LIGHTPANDA_EXECUTABLE_PATH version
command -v lightpanda

Troubleshoot common issues as follows:

  • No tasks are pulled: check server, region_id, ak, and sk, and confirm that datakit_dialtesting_task_number{protocol="BROWSER"} is greater than 0.
  • BROWSER tasks are available in the console but not executed on the node: confirm that [inputs.dialtesting.browser].enabled = false is not explicitly configured, and check whether the DataKit log contains browser.enabled is false or unsupported.
  • Results are not reported: check that task post_url is reachable, and that datakit_dialtesting_dataway_send_failed_number, datakit_dialtesting_worker_cached_points_number, and datakit_dialtesting_worker_dropped_points_number do not keep increasing.
  • Browser fails to start: confirm that the node's engine and engine_path match the installed browser and that the DataKit account can read and execute it. When using environment variables, check that the matching LIGHTPANDA_EXECUTABLE_PATH or CHROME_EXECUTABLE_PATH reaches the service environment.
  • Chrome dependencies are missing: follow the runtime requirements, check ldd, and install packages for the current distribution. After the version command succeeds, verify an actual dialtesting task.
  • Chrome reports No usable sandbox!: allow_no_sandbox = true allows one retry without the sandbox, with a warning; false fails explicitly. To retain the sandbox, check the non-root account, user namespaces, and host AppArmor or container seccomp policy.
  • Chinese screenshots show boxes or unexpected layout: check system Chinese fonts and enable load_fonts for pages that require web fonts.
  • Image selection: use the regular datakit:<version> image, which bundles both Lightpanda and official Chrome Headless Shell, and select the engine through browser configuration.
  • Screenshot is not uploaded: confirm that advance_options.screenshot_on_failure is true, then check screenshot_upload_error. Browser startup failures, process exits, and overall task timeouts may not leave a live session to capture.

Normally, the node can pull BROWSER tasks, datakit_dialtesting_worker_send_points_number{status="ok"} keeps increasing, and datakit_dialtesting_dataway_send_failed_number, datakit_dialtesting_worker_cached_points_number, and datakit_dialtesting_worker_dropped_points_number do not keep increasing.