Enabling the Java Profiler

The profiler is shipped within Datadog tracing libraries. If you are already using APM to collect traces for your application, you can skip installing the library and go directly to enabling the profiler.

Requirements

As of dd-trace-java version 1.0.0, you have two options for the engine that generates CPU profile data for Java applications: Java Flight Recorder (JFR) or the Datadog Profiler. As of version 1.7.0, Datadog Profiler is the default. Each profiler engine has different side effects, requirements, available configurations, and limitations, and this page describes each. You can enable either one or both engines. Enabling both captures both profile types at the same time.

All JVM-based languages, such as Java, Scala, Groovy, Kotlin, and Clojure are supported.

Continuous Profiler is not supported on serverless platforms, such as AWS Lambda.

Installation

To begin profiling applications:

1. If you are already using Datadog, upgrade your Agent to version 7.20.2+ or 6.20.2+. If you don’t have APM enabled to set up your application to send data to Datadog, in your Agent, set the DD_APM_ENABLED environment variable to true and listening to the port 8126/TCP.

2. Download dd-java-agent.jar, which contains the Java Agent class files:

   wget -O dd-java-agent.jar 'https://dtdg.co/latest-java-tracer'
   

   Note: Profiler is available in the dd-java-agent.jar library in versions 0.55+.

3. Enable the profiler by setting -Ddd.profiling.enabled flag or DD_PROFILING_ENABLED environment variable to true. Specify dd.service, dd.env, and dd.version so you can filter and group your profiles across these dimensions:

   Invoke your service:

   java \
       -javaagent:dd-java-agent.jar \
       -Ddd.service=<YOUR_SERVICE> \
       -Ddd.env=<YOUR_ENVIRONMENT> \
       -Ddd.version=<YOUR_VERSION> \
       -Ddd.profiling.enabled=true \
       -XX:FlightRecorderOptions=stackdepth=256 \
       -jar <YOUR_SERVICE>.jar <YOUR_SERVICE_FLAGS>
   

   export DD_SERVICE=<YOUR_SERVICE>
   export DD_ENV=<YOUR_ENV>
   export DD_VERSION=<YOUR_VERSION>
   export DD_PROFILING_ENABLED=true
   java \
       -javaagent:dd-java-agent.jar \
       -XX:FlightRecorderOptions=stackdepth=256 \
       -jar <YOUR_SERVICE>.jar <YOUR_SERVICE_FLAGS>
   

   Note: The -javaagent argument needs to be before -jar, adding it as a JVM option rather than an application argument. For more information, see the Oracle documentation:

   # Good:
   java -javaagent:dd-java-agent.jar ... -jar my-service.jar -more-flags
   # Bad:
   java -jar my-service.jar -javaagent:dd-java-agent.jar ...
   

4. After a minute or two, you can visualize your profiles on the Datadog APM > Profiling page.

Enabling CPU profiler engine options

Since dd-trace-java version 1.5.0, you have two options for the CPU profiler used, Datadog or Java Flight Recorder (JFR). Since version 1.7.0, Datadog is the default, but you can also optionally enable JFR for CPU profiling. You can enable either one or both engines. Enabling both captures both profile types at the same time.

The Datadog profiler records the active span on every sample, which improves the fidelity of the Code Hotspots and Endpoint profiling features. Enabling this engine supports much better integration with APM tracing.

The Datadog profiler consists of several profiling engines, including CPU, wallclock, allocation, and memory leak profilers.

export DD_PROFILING_DDPROF_ENABLED=true # this is the default in v1.7.0+
export DD_PROFILING_DDPROF_CPU_ENABLED=true

-Ddd.profiling.ddprof.enabled=true # this is the default in v1.7.0+
-Ddd.profiling.ddprof.cpu.enabled=true

sudo sh -c 'echo 2 >/proc/sys/kernel/perf_event_paranoid'

export DD_PROFILING_DDPROF_CPU_ENABLED=false

-Ddd.profiling.ddprof.cpu.enabled=false

Datadog profiler wallclock engine

The wallclock profiling engine is useful for profiling latency and integrates tightly with APM tracing. The engine samples all threads, on- or off-CPU, with active tracing activity and can be used to diagnose trace or span latency. The engine has been enabled by default since 1.7.0.

or:

-Ddd.profiling.ddprof.enabled=true # this is the default in v1.7.0+
-Ddd.profiling.ddprof.wall.enabled=true

For version 1.7.0+, to disable the wallclock profiler:

export DD_PROFILING_DDPROF_WALL_ENABLED=false

or:

-Ddd.profiling.ddprof.wall.enabled=false

For JMC users, the datadog.MethodSample event is emitted for wallclock samples.

The wallclock engine does not depend on the /proc/sys/kernel/perf_event_paranoid setting.

Datadog profiler allocation engine

In dd-java-agent v0.84.0+ and Java 15 and lower, the allocation profiler is turned off by default because it can use excessive CPU in allocation-heavy applications. This isn’t common, so you may want to try it in a staging environment to see if it affects your application. To enable it, see Enabling the allocation profiler.

The Datadog allocation profiling engine contextualizes allocation profiles, which supports allocation profiles filtered by endpoint. It is disabled by default, but you can enable it with:

export DD_PROFILING_DDPROF_ENABLED=true # this is the default in v1.7.0+
export DD_PROFILING_DDPROF_ALLOC_ENABLED=true

or:

-Ddd.profiling.ddprof.enabled=true # this is the default in v1.7.0+
-Ddd.profiling.ddprof.alloc.enabled=true

For JMC users, the Datadog allocation events are datadog.ObjectAllocationInNewTLAB and datadog.ObjectAllocationOutsideTLAB. The JFR allocation events are jdk.ObjectAllocationInNewTLAB and jdk.ObjectAllocationOutsideTLAB.

The allocation engine does not depend on the /proc/sys/kernel/perf_event_paranoid setting.

Collecting native stack traces

If the Datadog profiler CPU or wallclock engines are enabled, you can collect native stack traces. Native stack traces include things like JVM internals, native libraries used by your application or the JVM, and syscalls.

To enable native stack trace collection, understanding that it can destabilize your application, set:

export DD_PROFILING_DDPROF_ENABLED=true # this is the default in v1.7.0+
export DD_PROFILING_DDPROF_CSTACK=dwarf

or:

-Ddd.profiling.ddprof.enabled=true # this is the default in v1.7.0+
-Ddd.profiling.ddprof.cstack=dwarf

Configuration

You can configure the profiler using the following environment variables:

Not sure what to do next?

The Getting Started with Profiler guide takes a sample service with a performance problem and shows you how to use Continuous Profiler to understand and fix the problem.

Further Reading

Documentation, liens et articles supplémentaires utiles:

Getting Started with ProfilerDOCUMENTATION Learn more about available profile typesDOCUMENTATION Fix problems you encounter while using the profilerDOCUMENTATION