Enabling the Python Profiler

Enabling the Python Profiler

The profiler is shipped within Datadog tracing libraries. If you are already using APM to collect traces for your application, you can skip installing the library and go directly to enabling the profiler.

Requirements

The Datadog Profiler requires Python 2.7+ and Agent version 7.20.2+ or 6.20.2+.

The following profiling features are available depending on your Python version:

FeatureSupported Python versions
Wall time profilingPython >= 2.7
CPU time profilingPython >= 2.7 on POSIX platforms
Exception profilingPython >= 3.7 on POSIX platforms
Lock profilingPython >= 2.7
Memory profilingPython >= 3.5

Installation

Install ddtrace, which provides both tracing and profiling functionalities:

pip install ddtrace

Note: Profiling requires the ddtrace library version 0.40+.

If you are using a platform where ddtrace binary distribution is not available, install a development environment.

For example, on Alpine Linux, this can be done with:

apk install gcc musl-dev linux-headers

Usage

To automatically profile your code, set the DD_PROFILING_ENABLED environment variable to true when you use ddtrace-run:

DD_PROFILING_ENABLED=true \
DD_ENV=prod \
DD_SERVICE=my-web-app \
DD_VERSION=1.0.3 \
ddtrace-run python app.py

It is strongly recommended that you add tags like service or version, as they provide the ability to slice and dice your profiles across these dimensions. See Configuration below.

After a couple of minutes, visualize your profiles on the Datadog APM > Profiler page.

If you want to manually control the lifecycle of the profiler, use the ddtrace.profiling.profiler.Profiler object:

from ddtrace.profiling import Profiler

prof = Profiler(
    env="prod",  # if not specified, falls back to environment variable DD_ENV
    service="my-web-app",  # if not specified, falls back to environment variable DD_SERVICE
    version="1.0.3",   # if not specified, falls back to environment variable DD_VERSION
)
prof.start()

Caveats

When your process forks using os.fork, the profiler is actually stopped in the child process and needs to be restarted. For Python 3.7+ on Unix platforms, a new profiler is automatically started.

If you use Python < 3.7, or run on a non-Unix platform, you need to manually start a new profiler in your child process.

# For ddtrace-run users, call this in your child process
ddtrace.profiling.auto.start_profiler()

# Alternatively, for manual instrumentation,
# create a new profiler in your child process:
from ddtrace.profiling import Profiler

prof = Profiler()
prof.start()

Configuration

You can configure the profiler using the following environment variables:

Environment VariableKeyword Argument to ProfilerTypeDescription
DD_PROFILING_ENABLEDBooleanSet to true to enable profiler.
DD_PROFILING_HEAP_ENABLEDBooleanSet to true to enable memory heap profiling. (ddtrace 0.50+)
DD_SERVICEserviceStringThe Datadog service name.
DD_ENVenvStringThe Datadog environment name, for example, production.
DD_VERSIONversionStringThe version of your application.
DD_TAGStagsString / DictionaryTags to apply to an uploaded profile. If set with an environment variable, it must be a list of <key>:<value> separated by commas such as: layer:api,team:intake. If set with keyword argument, it must be a dictionary where keys are tag names and values are tag values such as:{"layer": "api", "team": "intake"}.

Not sure what to do next?

The Getting Started with Profiler guide takes a sample service with a performance problem and shows you how to use Continuous Profiler to understand and fix the problem.

Further Reading