Overview

Unified service tagging ties Datadog telemetry together by using three reserved tags: env, service, and version.

With these three tags, you can:

• Identify deployment impact with trace and container metrics filtered by version
• Navigate seamlessly across traces, metrics, and logs with consistent tags
• View service data based on environment or version in a unified fashion

Notes:

• The version tag is expected to change with each new application deployment. Two different versions of your application’s code should have distinct version tags.
• The official service of a log defaults to the container short-image if no Autodiscovery logs configuration is present. To override the official service of a log, add Autodiscovery Docker labels/pod annotations. For example: "com.datadoghq.ad.logs"='[{"service": "service-name"}]'
• Host information is excluded for database and cache spans because the host associated with the span is not the database/cache host.

Requirements

• Unified service tagging requires the setup of a Datadog Agent that is 6.19.x/7.19.x or higher.

• Unified service tagging requires a tracer version that supports new configurations of the reserved tags. More information can be found per language in the setup instructions.

• Unified service tagging requires knowledge of configuring tags. If you are unsure of how to configure tags, read the Getting Started with Tagging and Assigning Tags documentation before proceeding to configuration.

Configuration

To start configuring unified service tagging, choose your environment:

• Containerized
• Non-Containerized
• Serverless
• OpenTelemetry

Containerized environment

In containerized environments, env, service, and version are set through the service’s environment variables or labels (for example, Kubernetes deployment and pod labels, Docker container labels). The Datadog Agent detects this tagging configuration and applies it to the data it collects from containers.

To setup unified service tagging in a containerized environment:

1. Enable Autodiscovery. This allows the Datadog Agent to automatically identify services running on a specific container and gathers data from those services to map environment variables to the env, service, and version tags.

2. If you are using Docker, make sure the Agent can access your container’s Docker socket. This allows the Agent detect the environment variables and map them to the standard tags.

3. Configure your environment that corresponds to your container orchestration service based on either full configuration or partial configuration as detailed below.

Configuration

apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    tags.datadoghq.com/env: "<ENV>"
    tags.datadoghq.com/service: "<SERVICE>"
    tags.datadoghq.com/version: "<VERSION>"
...
template:
  metadata:
    labels:
      tags.datadoghq.com/env: "<ENV>"
      tags.datadoghq.com/service: "<SERVICE>"
      tags.datadoghq.com/version: "<VERSION>"
  containers:
  -  ...
     env:
          - name: DD_ENV
            valueFrom:
              fieldRef:
                fieldPath: metadata.labels['tags.datadoghq.com/env']
          - name: DD_SERVICE
            valueFrom:
              fieldRef:
                fieldPath: metadata.labels['tags.datadoghq.com/service']
          - name: DD_VERSION
            valueFrom:
              fieldRef:
                fieldPath: metadata.labels['tags.datadoghq.com/version']

template:
  metadata:
    labels:
      tags.datadoghq.com/env: "<ENV>"
      tags.datadoghq.com/service: "<SERVICE>"
      tags.datadoghq.com/version: "<VERSION>"

tags.datadoghq.com/<container-name>.env
tags.datadoghq.com/<container-name>.service
tags.datadoghq.com/<container-name>.version

apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    tags.datadoghq.com/env: "<ENV>"
    tags.datadoghq.com/service: "<SERVICE>"
    tags.datadoghq.com/version: "<VERSION>"
spec:
  template:
    metadata:
      labels:
        tags.datadoghq.com/env: "<ENV>"
        tags.datadoghq.com/service: "<SERVICE>"
        tags.datadoghq.com/version: "<VERSION>"

containers:
-  ...
    env:
        - name: DD_ENV
          valueFrom:
            fieldRef:
              fieldPath: metadata.labels['tags.datadoghq.com/env']
        - name: DD_SERVICE
          valueFrom:
            fieldRef:
              fieldPath: metadata.labels['tags.datadoghq.com/service']
        - name: DD_VERSION
          valueFrom:
            fieldRef:
              fieldPath: metadata.labels['tags.datadoghq.com/version']

ENV DD_SERVICE <SERVICE>
ENV DD_VERSION <VERSION>

LABEL com.datadoghq.tags.service="<SERVICE>"
LABEL com.datadoghq.tags.version="<VERSION>"

docker run -e DD_ENV=<ENV> -l com.datadoghq.tags.env=<ENV> ...

docker run -e DD_ENV="<ENV>" \
           -e DD_SERVICE="<SERVICE>" \
           -e DD_VERSION="<VERSION>" \
           -l com.datadoghq.tags.env="<ENV>" \
           -l com.datadoghq.tags.service="<SERVICE>" \
           -l com.datadoghq.tags.version="<VERSION>" \
           ...

com.datadoghq.tags.env
com.datadoghq.tags.service
com.datadoghq.tags.version

"environment": [
  {
    "name": "DD_ENV",
    "value": "<ENV>"
  },
  {
    "name": "DD_SERVICE",
    "value": "<SERVICE>"
  },
  {
    "name": "DD_VERSION",
    "value": "<VERSION>"
  }
],
"dockerLabels": {
  "com.datadoghq.tags.env": "<ENV>",
  "com.datadoghq.tags.service": "<SERVICE>",
  "com.datadoghq.tags.version": "<VERSION>"
}

"dockerLabels": {
  "com.datadoghq.tags.env": "<ENV>",
  "com.datadoghq.tags.service": "<SERVICE>",
  "com.datadoghq.tags.version": "<VERSION>"
}

Non-containerized environment

Depending on how you build and deploy your services’ binaries or executables, you may have several options available for setting environment variables. Since you may run one or more services per host, Datadog recommends scoping these environment variables to a single process.

To form a single point of configuration for all telemetry emitted directly from your services’ runtime for traces, logs, RUM resources, Synthetics tests, StatsD metrics, or system metrics, either:

1. Export the environment variables in the command for your executable:

   DD_ENV=<env> DD_SERVICE=<service> DD_VERSION=<version> /bin/my-service
   

2. Or use Chef, Ansible, or another orchestration tool to populate a service’s systemd or initd configuration file with the DD environment variables. When the service process starts, it has access to those variables.

   When configuring your traces for unified service tagging:

   1. Configure the APM Tracer with DD_ENV to keep the definition of env closer to the application that is generating the traces. This method allows the env tag to be sourced automatically from a tag in the span metadata.

   2. Configure spans with DD_VERSION to add version to all spans that fall under the service that belongs to the tracer (generally DD_SERVICE). This means that if your service creates spans with the name of an external service, those spans do not receive version as a tag.

      As long as version is present in spans, it is added to trace metrics generated from those spans. The version can be added manually in-code or automatically by the APM Tracer. When configured, these are used by the APM and DogStatsD clients to tag trace data and StatsD metrics with env, service, and version. If enabled, the APM tracer also injects the values of these variables into your logs.

      Note: There can only be one service per span. Trace metrics generally have a single service as well. However, if you have a different service defined in your hosts’ tags, that configured service tag shows up on all trace metrics emitted from that host.

   If you’re using connected logs and traces, enable automatic logs injection if supported for your APM Tracer. Then, the APM Tracer automatically injects env, service, and version into your logs, therefore eliminating manual configuration for those fields elsewhere.

   If you’re using connected RUM and traces, specify the browser application in the service field, define the environment in the env field, and list the versions in the version field of your initialization file.

   When you create a RUM application, confirm the env and service names.

   If you’re using connected Synthetic browser tests and traces, specify a URL to send headers to under the APM Integration for Browser Tests section of the Integration Settings page.

   You can use * for wildcards, for example: https://*.datadoghq.com.

   Tags are added in an append-only fashion for custom StatsD metrics. For example, if you have two different values for env, the metrics are tagged with both environments. There is no order in which one tag overrides another of the same name.

   If your service has access to DD_ENV, DD_SERVICE, and DD_VERSION, then the DogStatsD client automatically adds the corresponding tags to your custom metrics.

   Note: The Datadog DogStatsD clients for .NET and PHP do not support this functionality.

   You can add env and service tags to your infrastructure metrics. In non-containerized contexts, tagging for service metrics is configured at the Agent level.

   Because this configuration does not change for each invocation of a service’s process, adding version is not recommended.

   Single service per host

   Set the following configuration in the Agent’s main configuration file:

   env: <ENV>
   tags:
     - service:<SERVICE>
   

   This setup guarantees consistent tagging of env and service for all data emitted by the Agent.

   Multiple services per host

   Set the following configuration in the Agent’s main configuration file:

   env: <ENV>
   

   To get unique service tags on CPU, memory, and disk I/O metrics at the process level, configure a process check in the Agent’s configuration folder (for example, in the conf.d folder under process.d/conf.yaml):

   init_config:
   instances:
       - name: web-app
         search_string: ["/bin/web-app"]
         exact_match: false
         service: web-app
       - name: nginx
         search_string: ["nginx"]
         exact_match: false
         service: nginx-web-app
   

   Note: If you already have a service tag set globally in your Agent’s main configuration file, the process metrics are tagged with two services. Since this can cause confusion with interpreting the metrics, it is recommended to configure the service tag only in the configuration of the process check.

Serverless environment

For more information about AWS Lambda functions, see how to connect your Lambda telemetry using tags.

OpenTelemetry

When using OpenTelemetry, map the following resource attributes to their corresponding Datadog conventions:

export OTEL_RESOURCE_ATTRIBUTES="service.name=my-service,deployment.environment=production,service.version=1.2.3"

from opentelemetry.sdk.resources import Resource
from opentelemetry.sdk.trace import TracerProvider

resource = Resource(attributes={
   "service.name": "<SERVICE>",
   "deployment.environment": "<ENV>",
   "service.version": "<VERSION>"
})
tracer_provider = TracerProvider(resource=resource)

processors:
  transform:
    trace_statements:
      - context: resource
        statements:
          - set(attributes["service.name"], "my-service")
          - set(attributes["deployment.environment"], "production")
          - set(attributes["service.version"], "1.2.3")
...

Further Reading

Additional helpful documentation, links, and articles:

Learn how to use tags in the Datadog appDOCUMENTATION Use Version tags within Datadog APM to monitor deploymentsDOCUMENTATION Learn more about AutodiscoveryBLOG