The profiler is shipped within Datadog tracing libraries. If you are already using APM to collect traces for your application, you can skip installing the library and go directly to enabling the profiler.

Requirements

For a summary of the minimum and recommended runtime and tracer versions across all languages, read Supported Language and Tracer Versions.

The Datadog Profiler requires Go 1.19+.

For Code Hotspots and Endpoint Profiling, use dd-trace-go version 1.37.0+.

Continuous Profiler is not supported on serverless platforms, such as AWS Lambda.

Installation

To begin profiling applications:

1. Ensure Datadog Agent v6+ is installed and running. Datadog recommends using Datadog Agent v7+.

2. Get dd-trace-go using the command:

   go get gopkg.in/DataDog/dd-trace-go.v1/profiler
   

   Note: Profiler is available in the dd-trace-go library for versions 1.23.0+.

3. Import the profiler at the start of your application:

   import "gopkg.in/DataDog/dd-trace-go.v1/profiler"
   

4. Add the following snippet to start the profiler:

   err := profiler.Start(
       profiler.WithService("<SERVICE_NAME>"),
       profiler.WithEnv("<ENVIRONMENT>"),
       profiler.WithVersion("<APPLICATION_VERSION>"),
       profiler.WithTags("<KEY1>:<VALUE1>", "<KEY2>:<VALUE2>"),
       profiler.WithProfileTypes(
         profiler.CPUProfile,
         profiler.HeapProfile,
         // The profiles below are disabled by default to keep overhead
         // low, but can be enabled as needed.
   
         // profiler.BlockProfile,
         // profiler.MutexProfile,
         // profiler.GoroutineProfile,
       ),
   )
   if err != nil {
       log.Fatal(err)
   }
   defer profiler.Stop()
   

5. Optional: Enable the timeline feature (beta), see prerequisites.

6. Optional: Set up Source Code Integration to connect your profiling data with your Git repositories.

7. After a minute or two, visualize your profiles in the Datadog APM > Profiler page.

Note: By default, only the CPU and Heap profiles are enabled. Use profiler.WithProfileTypes to enable additional profile types.

Configuration

You can set profiler parameters in code with these functions:

Alternatively you can set profiler configuration using environment variables:

Showing C function calls in CPU profiles

By default, Go’s CPU profiler only shows detailed information for Go code. If your program calls C code, the time spent running C code is reflected in the profile, but the call stacks only show Go function calls.

To add detailed C function call information to CPU profiles, you may opt to use library such as ianlancetaylor/cgosymbolizer. To use this library:

1. Download the package:

   go get github.com/ianlancetaylor/cgosymbolizer@latest
   

2. Add the following import anywhere in your program:

   import _ "github.com/ianlancetaylor/cgosymbolizer"
   

Note: This library is considered experimental. It can cause (infrequent) deadlocks in programs that use C++ exceptions, or that use libraries such as tcmalloc, which also collect call stacks.

Save up to 14% CPU in production with PGO

Starting Go 1.21, the Go compiler supports Profile-Guided Optimization (PGO). PGO enables additional optimizations on code identified as hot by CPU profiles of production workloads. This is compatible with Datadog Go Continuous Profiler and can be used for production builds.

Follow this guide to set it up.

Not sure what to do next?

The Getting Started with Profiler guide takes a sample service with a performance problem and shows you how to use Continuous Profiler to understand and fix the problem.

Further Reading

Additional helpful documentation, links, and articles:

Getting Started with ProfilerDOCUMENTATION Learn more about available profile visualizationsDOCUMENTATION Fix problems you encounter while using the profilerDOCUMENTATION