The profiler is shipped within Datadog tracing libraries. If you are already using APM to collect traces for your application, you can skip installing the library and go directly to enabling the profiler.
Requirements
For a summary of the minimum and recommended runtime and tracer versions across all languages, read Supported Language and Tracer Versions.
The Datadog Profiler requires Go 1.19+.
For Code Hotspots and Endpoint Profiling, use dd-trace-go
version 1.37.0+.
Continuous Profiler is not supported on some serverless platforms, such as AWS Lambda.
Installation
To begin profiling applications:
Ensure Datadog Agent v6+ is installed and running. Datadog recommends using Datadog Agent v7+.
Get dd-trace-go
using the command:
go get gopkg.in/DataDog/dd-trace-go.v1/profiler
Note: Profiler is available in the dd-trace-go
library for versions 1.23.0+.
Import the profiler at the start of your application:
import "gopkg.in/DataDog/dd-trace-go.v1/profiler"
Add the following snippet to start the profiler:
err := profiler.Start(
profiler.WithService("<SERVICE_NAME>"),
profiler.WithEnv("<ENVIRONMENT>"),
profiler.WithVersion("<APPLICATION_VERSION>"),
profiler.WithTags("<KEY1>:<VALUE1>", "<KEY2>:<VALUE2>"),
profiler.WithProfileTypes(
profiler.CPUProfile,
profiler.HeapProfile,
// The profiles below are disabled by default to keep overhead
// low, but can be enabled as needed.
// profiler.BlockProfile,
// profiler.MutexProfile,
// profiler.GoroutineProfile,
),
)
if err != nil {
log.Fatal(err)
}
defer profiler.Stop()
Optional: Enable the timeline feature, see prerequisites.
Optional: Set up Source Code Integration to connect your profiling data with your Git repositories.
After a minute or two, visualize your profiles in the Datadog APM > Profiler page.
Note: By default, only the CPU and Heap profiles are enabled. Use profiler.WithProfileTypes to enable additional profile types.
If you automatically instrument your Go application with Orchestrion, it adds the continuous profiler code to your application. To enable the profiler at run time, set the environment variable DD_PROFILING_ENABLED=true
.
Configuration
You can set profiler parameters in code with these functions:
Function | Type | Description |
---|
WithService | String | The Datadog service name, for example, my-web-app . |
WithEnv | String | The Datadog environment name, for example, production . |
WithVersion | String | The version of your application. |
WithTags | List of strings | A list of tags to apply to an uploaded profile. Tags must be of the format <KEY>:<VALUE> . |
Alternatively you can set profiler configuration using environment variables:
Environment variable | Type | Description |
---|
DD_ENV | String | The environment name, for example, production . |
DD_SERVICE | String | The service name, for example, web-backend . |
DD_VERSION | String | The version of your service. |
DD_TAGS | String | Tags to apply to an uploaded profile. Must be a list of <key>:<value> separated by commas such as: layer:api,team:intake . |
Showing C function calls in CPU profiles
By default, Go’s CPU profiler only shows detailed information for Go code. If your program calls C code, the time spent running C code is reflected in the profile, but the call stacks only show Go function calls.
To add detailed C function call information to CPU profiles, you may opt to use library such as ianlancetaylor/cgosymbolizer. To use this library:
Download the package:
go get github.com/ianlancetaylor/cgosymbolizer@latest
Add the following import anywhere in your program:
import _ "github.com/ianlancetaylor/cgosymbolizer"
Note: This library is considered experimental. It can cause (infrequent) deadlocks in programs that use C++ exceptions, or that use libraries such as tcmalloc
, which also collect call stacks.
Save up to 14% CPU in production with PGO
Starting Go 1.21, the Go compiler supports Profile-Guided Optimization (PGO). PGO enables additional optimizations on code identified as hot by CPU profiles of production workloads. This is compatible with Datadog Go Continuous Profiler and can be used for production builds.
Follow this guide to set it up.
Not sure what to do next?
The Getting Started with Profiler guide takes a sample service with a performance problem and shows you how to use Continuous Profiler to understand and fix the problem.
Further Reading
Additional helpful documentation, links, and articles: