The profiler is shipped within Datadog tracing libraries. If you are already using APM to collect traces for your application, you can skip installing the library and go directly to enabling the profiler.

Requirements

For a summary of the minimum and recommended runtime and tracer versions across all languages, read Supported Language and Tracer Versions.

As of dd-trace-java 1.0.0, you have two options for the engine that generates profile data for Java applications: Java Flight Recorder (JFR) or the Datadog Profiler. As of dd-trace-java 1.7.0, Datadog Profiler is the default. Each profiler engine has different side effects, requirements, available configurations, and limitations, and this page describes each. You can enable either one or both engines. Enabling both captures both profile types at the same time.

All JVM-based languages, such as Java, Scala, Groovy, Kotlin, and Clojure are supported.

Continuous Profiler is not supported on serverless platforms, such as AWS Lambda.

Installation

To begin profiling applications:

1. Ensure Datadog Agent v6+ is installed and running. Datadog recommends using Datadog Agent v7+. If you don’t have APM enabled to set up your application to send data to Datadog, in your Agent, set the DD_APM_ENABLED environment variable to true and listening to the port 8126/TCP.

2. Download dd-java-agent.jar, which contains the Java Agent class files:

wget -O dd-java-agent.jar 'https://dtdg.co/latest-java-tracer'

curl -Lo dd-java-agent.jar 'https://dtdg.co/latest-java-tracer'

ADD 'https://dtdg.co/latest-java-tracer' dd-java-agent.jar

 **Note**: Profiler is available in the `dd-java-agent.jar` library in versions 0.55+.

3. Enable the profiler by setting -Ddd.profiling.enabled flag or DD_PROFILING_ENABLED environment variable to true. Specify dd.service, dd.env, and dd.version so you can filter and group your profiles across these dimensions:

   Invoke your service:

   java \
       -javaagent:dd-java-agent.jar \
       -Ddd.service=<YOUR_SERVICE> \
       -Ddd.env=<YOUR_ENVIRONMENT> \
       -Ddd.version=<YOUR_VERSION> \
       -Ddd.profiling.enabled=true \
       -XX:FlightRecorderOptions=stackdepth=256 \
       -jar <YOUR_SERVICE>.jar <YOUR_SERVICE_FLAGS>
   

   export DD_SERVICE=<YOUR_SERVICE>
   export DD_ENV=<YOUR_ENV>
   export DD_VERSION=<YOUR_VERSION>
   export DD_PROFILING_ENABLED=true
   java \
       -javaagent:dd-java-agent.jar \
       -XX:FlightRecorderOptions=stackdepth=256 \
       -jar <YOUR_SERVICE>.jar <YOUR_SERVICE_FLAGS>
   

   Note: The -javaagent argument needs to be before -jar, adding it as a JVM option rather than an application argument. For more information, see the Oracle documentation:

   # Good:
   java -javaagent:dd-java-agent.jar ... -jar my-service.jar -more-flags
   # Bad:
   java -jar my-service.jar -javaagent:dd-java-agent.jar ...
   

4. After a minute or two, you can visualize your profiles on the Datadog APM > Profiling page.

Enabling CPU profiler engine options

Since dd-trace-java version 1.5.0, you have two options for the CPU profiler used, Datadog or Java Flight Recorder (JFR). Since version 1.7.0, Datadog is the default, but you can also optionally enable JFR for CPU profiling. You can enable either one or both engines. Enabling both captures both profile types at the same time.

The Datadog profiler records the active span on every sample, which improves the fidelity of the Code Hotspots and Endpoint profiling features. Enabling this engine supports much better integration with APM tracing.

The Datadog profiler consists of several profiling engines, including CPU, wallclock, allocation, and memory leak profilers.

export DD_PROFILING_DDPROF_ENABLED=true # this is the default in v1.7.0+
export DD_PROFILING_DDPROF_CPU_ENABLED=true

-Ddd.profiling.ddprof.enabled=true # this is the default in v1.7.0+
-Ddd.profiling.ddprof.cpu.enabled=true

sudo sh -c 'echo 2 >/proc/sys/kernel/perf_event_paranoid'

export DD_PROFILING_DDPROF_CPU_ENABLED=false

-Ddd.profiling.ddprof.cpu.enabled=false

Datadog profiler wallclock engine

The wallclock profiling engine is useful for profiling latency and integrates tightly with APM tracing. The engine samples all threads, on- or off-CPU, with active tracing activity and can be used to diagnose trace or span latency. The engine has been enabled by default since 1.7.0.

-Ddd.profiling.ddprof.enabled=true # this is the default in v1.7.0+
-Ddd.profiling.ddprof.wall.enabled=true

For version 1.7.0+, to disable the wallclock profiler:

export DD_PROFILING_DDPROF_WALL_ENABLED=false

or:

-Ddd.profiling.ddprof.wall.enabled=false

For JMC users, the datadog.MethodSample event is emitted for wallclock samples.

The wallclock engine does not depend on the /proc/sys/kernel/perf_event_paranoid setting.

Profiler allocation engine

export DD_PROFILING_ENABLED_EVENTS=jdk.ObjectAllocationInNewTLAB,jdk.ObjectAllocationOutsideTLAB

-Ddd.profiling.enabled.events=jdk.ObjectAllocationInNewTLAB,jdk.ObjectAllocationOutsideTLAB

export DD_PROFILING_DDPROF_ENABLED=true # this is the default in v1.7.0+
export DD_PROFILING_DDPROF_ALLOC_ENABLED=true # this is the default in v1.17.0+

-Ddd.profiling.ddprof.enabled=true # this is the default in v1.7.0+
-Ddd.profiling.ddprof.alloc.enabled=true # this is the default in v1.17.0+

Live-heap profiler engine

Since: v1.17.0. Requires JDK 11+.

The live-heap profiler engine is useful for investigating the overall memory usage of your service and identifying potential memory leaks. The engine samples allocations and keeps track of whether those samples survived the most recent garbage collection cycle. The number of surviving samples is used to estimate the number of live objects in the heap. The number of tracked samples is limited to avoid unbounded growth of the profiler’s memory usage.

The engine is disabled by default, but you can enable it with:

export DD_PROFILING_DDPROF_LIVEHEAP_ENABLED=true

or:

-Ddd.profiling.ddprof.liveheap.enabled=true

For JMC users, the Datadog live-heap event is datadog.HeapLiveObject.

The allocation engine does not depend on the /proc/sys/kernel/perf_event_paranoid setting.

Collecting native stack traces

If the Datadog profiler CPU or wallclock engines are enabled, you can collect native stack traces. Native stack traces include things like JVM internals, native libraries used by your application or the JVM, and syscalls.

To enable native stack trace collection, understanding that it can destabilize your application, set:

export DD_PROFILING_DDPROF_ENABLED=true # this is the default in v1.7.0+
export DD_PROFILING_DDPROF_CSTACK=dwarf

or:

-Ddd.profiling.ddprof.enabled=true # this is the default in v1.7.0+
-Ddd.profiling.ddprof.cstack=dwarf

Configuration

You can configure the profiler using the following environment variables:

Not sure what to do next?

The Getting Started with Profiler guide takes a sample service with a performance problem and shows you how to use Continuous Profiler to understand and fix the problem.

Further Reading

Additional helpful documentation, links, and articles:

Getting Started with ProfilerDOCUMENTATION Learn more about available profile visualizationsDOCUMENTATION Fix problems you encounter while using the profilerDOCUMENTATION