Single Step APM Instrumentation (Beta)

Requirements

  • Languages and architectures: Single step APM instrumentation only supports tracing Java, Python, Ruby, Node.js, and .NET Core services on x86_64 and arm64 architectures.

  • Operating systems: Linux VMs (Debian, Ubuntu, Amazon Linux, CentOS/Red Hat, Fedora), Docker, Kubernetes clusters with Linux containers.

Enabling APM on your services in one step

If you install or update a Datadog Agent with the Enable APM Instrumentation (beta) option selected, the Agent is installed and configured to enable APM. This allows you to automatically instrument your application, without any additional installation or configuration steps. Restart services for this instrumentation to take effect.

The following examples show how it works on each infrastructure type.

With one command, you can install, configure, and start the Agent, while also instrumenting your services with APM.

For an Ubuntu host:

  1. Run the one-line installation command:

    DD_API_KEY=<YOUR_DD_API_KEY> DD_SITE="<YOUR_DD_SITE>" DD_APM_INSTRUMENTATION_ENABLED=host bash -c "$(curl -L https://s3.amazonaws.com/dd-agent/scripts/install_script_agent7.sh)"
    

    a. Replace <YOUR_DD_API_KEY> with your Datadog API.

    b. Replace <YOUR_DD_SITE> with your Datadog site.

  2. Exit your current shell session.

  3. Start a new shell session.

  4. Restart the services on the host or VM.

  5. Explore the performance observability of your services in Datadog.

Specifying tracing library versions

By default, enabling APM on your server installs support for Java, Python, Ruby, Node.js, and .NET Core services. If you only have services implemented in some of these languages, set DD_APM_INSTRUMENTATION_LIBRARIES in your one-line installation command:

DD_APM_INSTRUMENTATION_LIBRARIES="java:1.25.0,python" DD_API_KEY=<YOUR_DD_API_KEY> DD_SITE="<YOUR_DD_SITE>" DD_APM_INSTRUMENTATION_ENABLED=host DD_ENV=staging bash -c "$(curl -L https://s3.amazonaws.com/dd-agent/scripts/install_script_agent7.sh)"

You can optionally provide a version number for the tracing library by placing a colon after the language name and specifying the tracing library version. If you don’t specify a version, it defaults to the latest version. Language names are comma-separated.

Supported languages include:

  • .NET (dotnet)
  • Python (python)
  • Java (java)
  • Node.js (js)
  • Ruby (ruby)

Note: For the Node.js tracing library, different versions of Node.js are compatible with different versions of the Node.js tracing library. See DataDog/dd-trace-js: JavaScript APM Tracer for more information.

Tagging observability data by environment

Set DD_ENV in your one-line installation command for Linux to automatically tag instrumented services and other telemetry that pass through the Agent with a specific environment. For example, if the Agent is installed in your staging environment, set DD_ENV=staging to associate your observability data with staging.

For example:

DD_API_KEY=<YOUR_DD_API_KEY> DD_SITE="<YOUR_DD_SITE>" DD_APM_INSTRUMENTATION_ENABLED=host DD_ENV=staging bash -c "$(curl -L https://s3.amazonaws.com/dd-agent/scripts/install_script_agent7.sh)"

For a Docker Linux container:

  1. Install the library injector:
    bash -c "$(curl -L https://s3.amazonaws.com/dd-agent/scripts/install_script_docker_injection.sh)"
    
  2. Configure the Agent in Docker:
    docker run -d --name dd-agent \
      -e DD_API_KEY=${YOUR_DD_API_KEY} \
      -e DD_APM_ENABLED=true \
      -e DD_APM_NON_LOCAL_TRAFFIC=true \
      -e DD_DOGSTATSD_NON_LOCAL_TRAFFIC=true \
      -e DD_APM_RECEIVER_SOCKET=/opt/datadog/apm/inject/run/apm.socket \
      -e DD_DOGSTATSD_SOCKET=/opt/datadog/apm/inject/run/dsd.socket \
      -v /opt/datadog/apm:/opt/datadog/apm \
      -v /var/run/docker.sock:/var/run/docker.sock:ro \
      gcr.io/datadoghq/agent:7
    
    Replace <YOUR_DD_API_KEY> with your Datadog API.
  3. Restart the Docker containers.
  4. Explore the performance observability of your services in Datadog.

Specifying tracing library versions

By default, enabling APM on your server installs support for Java, Python, Ruby, Node.js, and .NET services. If you only have services implemented in some of these languages, set DD_APM_INSTRUMENTATION_LIBRARIES when running the installation script.

For example, to install support for only v1.25.0 of the Java tracing library and the latest Python tracing library, add the following to the installation command:

DD_APM_INSTRUMENTATION_LIBRARIES="java:1.25.0,python" bash -c "$(curl -L https://s3.amazonaws.com/dd-agent/scripts/install_script_docker_injection.sh)"

You can optionally provide a version number for the tracing library by placing a colon after the language name and specifying the tracing library version. If you don’t specify a version, it defaults to the latest version. Language names are comma-separated.

Supported languages include:

  • .NET (dotnet)
  • Python (python)
  • Java (java)
  • Node.js (js)
  • Ruby (ruby)

Note: For the Node.js tracing library, different versions of Node.js are compatible with different versions of the Node.js tracing library. See DataDog/dd-trace-js: JavaScript APM Tracer for more information.

Tagging observability data by environment

Set DD_ENV in the library injector installation command for Docker to automatically tag instrumented services and other telemetry that pass through the Agent with a specific environment. For example, if the Agent is installed in your staging environment, set DD_ENV=staging to associate your observability data with staging.

For example:

docker run -d --name dd-agent \
  -e DD_API_KEY=${YOUR_DD_API_KEY} \
  -e DD_APM_ENABLED=true \
  -e DD_ENV=staging \
  -e DD_APM_NON_LOCAL_TRAFFIC=true \
  -e DD_DOGSTATSD_NON_LOCAL_TRAFFIC=true \
  -e DD_APM_RECEIVER_SOCKET=/opt/datadog/apm/inject/run/apm.socket \
  -e DD_DOGSTATSD_SOCKET=/opt/datadog/apm/inject/run/dsd.socket \
  -v /opt/datadog/apm:/opt/datadog/apm \
  -v /var/run/docker.sock:/var/run/docker.sock:ro \
  gcr.io/datadoghq/agent:7

You can enable APM by installing the Agent with the Datadog Helm chart. This deploys the Datadog Agent across all nodes in your Linux-based Kubernetes cluster with a DaemonSet.

Note: Single step instrumentation doesn’t instrument applications in the namespace where you install the Datadog Agent. It’s recommended to install the Agent in a separate namespace in your cluster where you don’t run your applications.

Requirements

  • Make sure you have Helm installed.

Installation

To enable single step instrumentation with Helm:

  1. Add the Helm Datadog repo:

     helm repo add datadog https://helm.datadoghq.com
     helm repo update
    
  2. Create a Kubernetes secret to store your Datadog API key:

    kubectl create secret generic datadog-secret --from-literal api-key=$DD_API_KEY
    
  3. Create datadog-values.yaml and add the following configuration:

    datadog:
     apiKeyExistingSecret: datadog-secret
     site: <DATADOG_SITE>
     apm:
       instrumentation:
          enabled: true
    

    Replace <DATADOG_SITE> with your Datadog site.

  4. Run the following command:

    helm install datadog-agent -f datadog-values.yaml datadog/datadog
    
  5. Do a rolling restart on your applications for instrumentation to take effect.

Enabling or disabling instrumentation for namespaces

You can choose to selectively instrument specific namespaces or choose to not instrument them.

To enable instrumentation for specific namespaces, replace enabled: true with enabledNamespaces configuration in your datadog-values.yaml file:

      datadog:
        apiKeyExistingSecret: datadog-secret
        site: <DATADOG_SITE>
        apm:
          instrumentation:
            enabledNamespaces: # Add namespaces to instrument
               - namespace_1
               - namespace_2
 
The enabled: true option enables instrumentation for the entire cluster. You need to remove this to only enable instrumentation for specific namespaces.

To disable instrumentation for specific namespaces, add the disabledNamespaces configuration to your datadog-values.yaml file:

   datadog:
     apiKeyExistingSecret: datadog-secret
     site: <DATADOG_SITE>
     apm:
       instrumentation:
         enabled: true
         disabledNamespaces: # Add namespaces to not instrument
            - namespace_1
            - namespace_2

Specifying tracing library versions

You can optionally set specific tracing library versions to use. If you don’t specify a version, it defaults to the latest version. To find the latest version for a library, go to Releases in the dd-trace-<language> GitHub repo. For example, dd-trace-dotnet releases.

To set specific tracing library versions, add the following configuration to your datadog-values.yaml file:

   datadog:
     apiKeyExistingSecret: datadog-secret
     site: <DATADOG_SITE>
     apm:
       instrumentation:
         enabled: true
         libVersions: # Add any versions you want to set
            dotnet: v2.46.0
            python: v1.20.6
            java: v1.22.0
            js: v4.17.0
            ruby: v1.15.0

Supported languages include:

  • .NET (dotnet)
  • Python (python)
  • Java (java)
  • Node.js (js)
  • Ruby (ruby)

Tagging observability data by environment

Automatically tag instrumented services and other telemetry that pass through the Agent with a specific environment. For example, if the Agent is installed in your staging environment, set env:staging to associate your observability data with staging.

For example, add the following configuration to your datadog-values.yaml file:

   datadog:
     apiKeyExistingSecret: datadog-secret
     site: <DATADOG_SITE>
     tags:
         - env:staging
     apm:
       instrumentation:
         enabled: true

Removing Single Step APM instrumentation from your Agent

If you don’t want to collect trace data for a particular service, host, VM, or container, complete the follow steps:

Removing instrumentation for specific services

Run the following commands and restart the service to stop injecting the library into the service and stop producing traces from that service.

  1. Add the DD_INSTRUMENT_SERVICE_WITH_APM environment variable to the service startup command:

    DD_INSTRUMENT_SERVICE_WITH_APM=false <service_start_command>
    
  2. Restart the service.

  1. Add the DD_INSTRUMENT_SERVICE_WITH_APM environment variable to the service startup command:
    docker run -e DD_INSTRUMENT_SERVICE_WITH_APM=false <service_start_command>
    
  2. Restart the service.
  1. Set the admission.datadoghq.com/enabled: label to "false" for the pod spec:
    spec:
      template:
        metadata:
          labels:
            admission.datadoghq.com/enabled: "false"
    
  2. Apply the configuration:
    kubectl apply -f /path/to/your/deployment.yaml
    
  3. Restart the services you want to remove instrumentation for.

Removing APM for all services on the infrastructure

To stop producing traces, remove library injectors and restart the infrastructure:

  1. Run:
    dd-host-install --uninstall
    
  2. Restart your host.
  1. Uninstall local library injection:
    dd-container-install --uninstall
    
  2. Restart Docker:
    systemctl restart docker
    
    Or use the equivalent for your environment.
  1. Under apm:, remove instrumentation: and all following configuration in datadog-values.yaml.
  2. Run the following command:
    helm upgrade datadog-agent -f datadog-values.yaml datadog/datadog
    
  3. Restart your services.

Further reading

Additional helpful documentation, links, and articles: