Flameshot

Flameshot is a lightweight automated profiling tool running in Sidecar mode. It monitors the resource usage (CPU/Memory) of target processes and automatically triggers underlying Profilers (such as async-profiler) when preset thresholds are reached, enabling non-intrusive on-site snapshot collection.

Core Concepts¶

Operating Mode¶

Flameshot is deployed using the Sidecar Container pattern. It must run in the same Pod as the main business container (Main Container) and have PID namespace sharing enabled.

1. Monitor: Flameshot continuously polls the resource levels of target processes within the main container.
2. Trigger: When thresholds are met (e.g., CPU > 80%) or an HTTP API request is received, a collection task is triggered.
3. Execute: Based on the configured language type (currently supporting Java), it invokes the corresponding Profiler tool to attach to the target process.
4. Collect: The generated Profile files (e.g., .jfr) are stored in a shared volume and subsequently uploaded to the data observability center.
5. Timed: After configuring FLAMESHOT_AUTO_PROFILING, it periodically collects profiling data for all matched processes. The sample duration defaults to 30 seconds and can be adjusted through FLAMESHOT_AUTO_PROFILING_DURATION.
6. OOM Summary: When a container oom_kill increment is detected, Flameshot tries to automatically parse -XX:+HeapDumpOnOutOfMemoryError and -XX:HeapDumpPath=... from the target Java process arguments. If the dump file is generated inside the shared volume, Flameshot finds the corresponding .hprof and uploads a summary log.
7. High-Watermark Lightweight Snapshots: When memory usage approaches the cgroup emergency threshold, Flameshot automatically runs jcmd GC.class_histogram and jcmd Thread.print, stores the raw output in the shared volume, and uploads lightweight summary logs.

Use Cases¶

• Production Safety Net: Automatically preserve on-site evidence before a service crashes due to CPU spikes or memory leaks.
• Performance Stress Test Analysis: Cooperate with stress testing platforms to automatically collect performance hotspots under high load.

Configuration¶

All Flameshot behaviors are controlled via environment variables. Configuration is divided into Global Settings and Profiling Policies.

Global Environment Variables¶

These variables control the basic behavior of the Sidecar container.

Profiling Policy Configuration¶

Target monitoring rules are defined via the FLAMESHOT_PROCESSES environment variable. The value must be a standard JSON Array string.

To maintain readability in Kubernetes YAML, it is strongly recommended to use YAML's block scalar syntax (|) for writing the JSON configuration, as shown below:

    env:
      # ... other environment variables ...
      - name: FLAMESHOT_PROCESSES
        value: |
          [
            {
              "service": "user-service",
              "language": "java",
              "command": "^java.*user-service\\.jar$",
              "duration": "60s",
              "events": "cpu,alloc",
              "cpu_usage_percent": 80,
              "mem_usage_percent": 80,
              "mem_usage_mb": 1024,
              "mem_usage_percent_emergency": 92,
              "mem_usage_mb_emergency": 1536,
              "emergency_duration": "10s",
              "tags": [
                "env:prod",
                "version:v1.2"
              ]
            }
          ]

Common Field Descriptions:

• service (String): Service name reported to the observability center.
• language (String): Target process language. Currently supports java.
• command (String): Regular expression to match the process command line.
• duration (String): Duration of a single collection (e.g., 30s, 1m). Note: To avoid execution timeouts, it is recommended not to exceed 5 minutes.
• emergency_duration (String): Shorter profiling duration used after an emergency memory hit. 10s or 15s is recommended.
• tags (List): List of custom tags; recommended to include meta-information like env, version.
• cpu_usage_percent (Int): CPU trigger threshold (0-N). Values may exceed 100 in multi-core environments.
• mem_usage_percent (Int): Average memory-percentage threshold (0-100), evaluated by the latest 5 points.
• mem_usage_mb (Int): Average RSS threshold in MB, evaluated by the latest 5 points.
• mem_usage_percent_emergency (Int): Instant emergency memory-percentage threshold (0-100). A single hit triggers immediately.
• mem_usage_mb_emergency (Int): Instant emergency RSS threshold in MB. A single hit triggers immediately.
• cpu_usage_percent, mem_usage_percent, and mem_usage_mb skip threshold checks when omitted or set to 0.
• When FLAMESHOT_POD_MEM_LIMIT is configured, mem_usage_percent and mem_usage_percent_emergency prefer Pod-limit memory percentage instead of host memory percentage.

Language Specifics¶

Flameshot invokes different underlying tools depending on the technology stack of the monitored application.

Deployment¶

Kubernetes Sidecar Deployment¶

For Flameshot to work correctly, the Pod configuration must meet the following three conditions:

1. Shared Process Namespace (shareProcessNamespace: true).
2. Shared Storage Volume (EmptyDir).
3. System Capabilities (Capabilities).

YAML Example:

apiVersion: v1
kind: Pod
metadata:
  name: java-app-profiled
spec:
  # 1. [Core] Enable PID sharing so Sidecar can see the Java process
  shareProcessNamespace: true

  volumes:
  - name: shared-data
    emptyDir: {}

  containers:
  # Business Container
  - name: my-app
    image: my-app:latest
    volumeMounts:
    - name: shared-data
      mountPath: /data # Must match Sidecar configuration

  # Flameshot Sidecar
  - name: flameshot
    image: pubrepo.jiagouyun.com/datakit/flameshot:latest
    env:
      - name: FLAMESHOT_PROFILING_PATH
        value: "/data"
      # ... other environment variables ...

    # 2. [Core] Grant ptrace capability
    securityContext:
      capabilities:
        add: ["SYS_PTRACE"]

    # 3. [Core] Mount the same directory
    volumeMounts:
    - name: shared-data
      mountPath: /data

OOM HProf Summary Requirements¶

If you want Flameshot to automatically recover .hprof summary information after a Java OOM, all of the following conditions must be met:

1. Enable -XX:+HeapDumpOnOutOfMemoryError in the application JVM arguments.
2. Configure -XX:HeapDumpPath=/data/... in the JVM arguments, and make sure the path is inside the shared volume.
3. Enable FLAMESHOT_OOM_HPROF_ENABLED=true for the Flameshot Sidecar.
4. It is recommended to set FLAMESHOT_OOM_HPROF_MATCH_WINDOW explicitly so the matching window is operationally unambiguous.

For example:

java \
  -XX:+HeapDumpOnOutOfMemoryError \
  -XX:HeapDumpPath=/data/dumps/app.hprof \
  -jar app.jar

Notes:

• Flameshot now discovers HeapDumpPath directly from the target Java process arguments. There is no separate configuration item for the .hprof path.
• FLAMESHOT_OOM_HPROF_ENABLED only enables the Flameshot-side recovery workflow; it does not inject HeapDump-related flags into the target JVM.
• If the target process does not enable HeapDumpOnOutOfMemoryError, or if HeapDumpPath is not inside a shared volume, Flameshot can only record the OOM event and cannot locate the .hprof file.
• If the container is terminated before the dump is fully written, .hprof may still be unavailable. In practice, high-watermark jcmd snapshots should be enabled together with OOM summary recovery.

High-Watermark JCmd Snapshots¶

When the cgroup memory usage approaches the emergency threshold, Flameshot runs the following lightweight commands in parallel:

1. jcmd <pid> GC.class_histogram
2. jcmd <pid> Thread.print

Behavior:

• The raw output is written into the shared directory configured by FLAMESHOT_PROFILING_PATH.
• The default jcmd timeout is 10s, and can be adjusted through FLAMESHOT_JCMD_TIMEOUT.
• It is recommended to declare FLAMESHOT_JCMD_SNAPSHOT_ENABLED=true explicitly in deployment manifests instead of relying on image or template defaults.
• FLAMESHOT_JCMD_SNAPSHOT_ENABLED=true only authorizes Flameshot to attempt jcmd; snapshots are still skipped when the Sidecar cannot access an executable jcmd, Attach prerequisites are missing, or runtime isolation prevents attachment.
• Example filenames:
• jcmd_gc_class_histogram_<pid>_<timestamp>.txt
• jcmd_thread_print_<pid>_<timestamp>.txt
• Flameshot also uploads a lightweight summary log for each snapshot.

Prerequisites:

1. Enable FLAMESHOT_JCMD_SNAPSHOT_ENABLED=true explicitly for the Flameshot Sidecar.
2. Make jcmd available to the Sidecar, either from the target JVM image or through a shared JDK path.
3. Enable shareProcessNamespace: true, and ensure the Sidecar has the permissions and shared runtime paths required by JVM Attach.

Summary log fields:

• snapshot_type: gc_class_histogram or thread_print
• output_path: Raw output file path
• output_size_bytes: Output file size
• preview: A condensed preview
• GC.class_histogram extracts the top few lines with the largest classes
• Thread.print extracts thread titles and thread state lines

Why this matters:

• These snapshots are much lighter than .hprof, so they are often more practical when the Pod is close to OOM but has not yet been killed.
• In direct kernel oom_kill scenarios, .hprof may never be generated, while high-watermark jcmd snapshots still have a chance to preserve useful evidence earlier.

Docker Local Testing¶

If you need to test in a local Docker environment, use the following command to start Flameshot and monitor the target container.

Prerequisites:

• The main container and Flameshot container must share /opt/java/openjdk (or the actual JDK path).
• Use --pid="container:<target_id>" or shared volumes (depending on the specific Docker version).

Test Image: pubrepo.jiagouyun.com/datakit/flameshot:1.85.1-testing_testing-iss-2876

Startup Command Example:

docker run -d \
  --name flameshot-debug \
  --volumes-from <YOUR_JAVA_APP_CONTAINER> \
  -e FLAMESHOT_DATAKIT_ADDR="http://datakit:9529/profiling/v1/input" \
  -e FLAMESHOT_PROCESSES='[{"service":"local-test","command":"java","language":"java","cpu_usage_percent":10}]' \
  pubrepo.jiagouyun.com/datakit/flameshot:1.85.1-testing_testing-iss-2876

API Reference¶

Flameshot provides an HTTP interface allowing users or automated O&M scripts to manually trigger collection tasks.

Manual Triggering¶

Interface Address: GET /v1/profile

Semantic Explanation: This interface is used to generate a Profile dataset on demand, not to retrieve monitoring metrics.

Request Parameters:

Usage Examples:

1. Trigger collection by PID:

   # Collect CPU and memory allocation data for the process with PID 1234 for 30 seconds
   curl "http://localhost:8089/v1/profile?pid=1234&duration=30s&events=cpu,alloc"
   

2. Trigger collection by process name regex:

   # Collect data for the process matching tmall.jar for the default duration
   curl "http://localhost:8089/v1/profile?command=^java\\b.*tmall\\.jar$"
   

JFR format¶

async-profiler events notes:

Troubleshooting¶

1. Cannot collect data?

   • Check if shareProcessNamespace: true is enabled in the Pod.
   • Check if the Sidecar has SYS_PTRACE capability.

2. File not uploaded?

   • Check if FLAMESHOT_PROFILING_PATH is correctly mounted between the two containers.
   • The system automatically manages file life cycles and will attempt to delete temporary files after collection is complete.

Changelog¶

0.2.1 (2026-2-11)¶

Optimize¶

• optimize
  • In a container environment, use the configured resource size as the base value for threshold calculation.

0.2.0 (2026-2-4)¶

New Features¶

• Add config
  • Support configuring scheduled Profiling execution via the environment variable FLAMESHOT_AUTO_PROFILING
• optimize
  • Optimize the threshold processing logic for configuration.

0.1.0 (2025-12-17)¶

The first official release of Flameshot, focusing on providing automated profiling capabilities for Java applications in containerized environments.

New Features¶

• Core Architecture:
  • Support for Kubernetes Sidecar Mode deployment, utilizing shared PID namespaces for non-intrusive monitoring.
  • Support for Linux AMD64 and ARM64 multi-architecture execution.
• Language Support:
  • Java: Deep integration with async-profiler, supporting various event collections like CPU, Alloc, Lock, etc.
  • Automatic detection and adaptation to the target container's JDK environment.
• Trigger Mechanism:
  • Threshold Trigger: Support for automatic triggering based on CPU usage (cpu_usage_percent) and memory usage/amount (mem_usage_percent/mem_usage_mb).
  • API Trigger: Provided HTTP interface GET /v1/monitor (Note: should be /v1/profile as per API section), supporting manual trigger by PID or regex process name matching.
• Data Integration:
  • Support for automatically reporting generated .jfr or flame graph data to DataKit.
  • Support for flexible multi-process monitoring policies and tags (tags) via the FLAMESHOT_PROCESSES environment variable.