Overview

Datadog uses OpenTelemetry resource attributes to match metrics, traces, and logs to hosts. When all telemetry from a host identifies the host the same way, you can correlate signals from that host, and the telemetry inherits the host’s tags.

For the hostname resolution order and the full list of supported resource attributes, see Mapping OpenTelemetry Semantic Conventions to Hostnames.

Hostname recommendations

The right configuration depends on how you send telemetry to Datadog. Find each setup in your telemetry path and follow its section. For example, if a node-level Collector sends telemetry through a gateway, follow the recommendations for both the node-level Collector and the gateway.

OTLP ingestion by the Datadog Agent

Deploy the Datadog Agent on every host that generates OTLP telemetry. Datadog doesn’t support sending telemetry from one host to an Agent on another host. For setup instructions, see OTLP Ingestion by the Datadog Agent.

If incoming telemetry has no valid hostname attributes, Datadog uses the Agent hostname. If you set host.name, host.id, or another hostname attribute, set it to the Agent hostname to avoid duplicate hosts. To override hostname resolution, set the datadog.host.name resource attribute to the Agent hostname.

DDOT Collector exporting directly to Datadog

The DDOT Collector’s infraattributes processor adds infrastructure attributes and tags to OTLP telemetry. By default, DDOT adds this processor to pipelines that use the Datadog Exporter. The processor needs resource attributes that identify the source container so it can look up infrastructure tags. For supported attributes and troubleshooting steps, see Infrastructure tags are missing from telemetry.

The rest of the configuration depends on whether the DDOT Collector runs on a host.

Host-based deployments

This applies to the DDOT Collector as a Kubernetes DaemonSet and on Linux and Windows hosts.

Because the DDOT Collector runs inside the Datadog Agent, hostname attributes on incoming telemetry can resolve to a different name than the Agent hostname. A single node then appears as two hosts. To use the Agent hostname instead, enable allow_hostname_override:

processors:
  infraattributes:
    allow_hostname_override: true

Add infraattributes to each pipeline’s processors list if it isn’t there already. DDOT automatically adds its own copy to pipelines that don’t list the processor, and that copy uses the default settings.

Fargate sidecar deployments

This applies to the DDOT Collector on ECS Fargate and EKS Fargate, where the Datadog Agent runs as a sidecar container in the same task or pod as your application.

Fargate does not support host-based deployments, so the host-based guidance does not apply. Instead, supply the platform resource attributes that infraattributes needs:

• On ECS Fargate, add the ECS resource detector to your OpenTelemetry SDK to provide the aws.ecs.task.arn attribute.
• On EKS Fargate, add the EKS resource detector to your SDK, or add the resource detection processor with the [env, eks] detectors to your DDOT Collector configuration. For an example, see the EKS Fargate installation guide.

OpenTelemetry Collector exporting to Datadog over OTLP

This applies to the recommended OpenTelemetry Collector setup, where the Collector sends telemetry to Datadog with the OTLP HTTP exporter.

Run one Collector on each host or Kubernetes node, and send telemetry to the Collector on the same host or node. The recommended configurations already set up host detection for each environment. If you adapt them, keep these settings:

• Hosts and VMs: The resource_detection processor with the system detector. On a cloud provider, add its detector.
• Docker: The Collector cannot detect the host from inside its container. Set host.name in the Collector’s OTEL_RESOURCE_ATTRIBUTES environment variable.
• Kubernetes: The k8s_attributes and resource_detection processors. On non-cloud Kubernetes, set k8s.node.name in the Collector’s OTEL_RESOURCE_ATTRIBUTES environment variable. On cloud Kubernetes, use the detector for your cloud provider instead of the environment variable.

To add host detection to a Collector configuration that doesn’t include it, see Configure hostname processors in the Collector.

The recommended setup’s span_metrics connector uses host attributes as dimensions, so APM trace metrics keep their host tags. If you adapt the configuration, keep all of its dimensions. For details, see Span metrics connector.

The recommended setup does not support serverless or task-based runtimes such as AWS Lambda, ECS Fargate, and EKS Fargate. For those, see Direct OTLP intake without a Collector.

Direct OTLP intake without a Collector

This applies when telemetry goes directly to Datadog OTLP intake, without a Datadog Agent or OpenTelemetry Collector. For example:

• Applications on hosts that export with an OpenTelemetry SDK
• Serverless platforms such as AWS Lambda, ECS Fargate, Azure Functions, and Cloud Run
• Managed platforms

Set the resource attributes for your environment before you export telemetry. Turn on your SDK’s built-in resource detectors when available, or set the attributes yourself. On serverless and managed platforms, identify workloads with platform resource attributes instead of host.name.

If you run the DDOT Collector as a sidecar on ECS Fargate or EKS Fargate, see Fargate sidecar deployments instead.

Collector exporting through a gateway

This applies to OpenTelemetry Collector gateway deployments and to the DDOT Collector as a gateway on Kubernetes.

In a gateway deployment, the Collector that exports to Datadog does not run on the host that produced the telemetry. If telemetry reaches the gateway without host information, telemetry from many hosts can collapse onto the gateway’s hostname, or each gateway pod can register as its own host.

Attach host information in the node-level Collector. Then configure the gateway to keep those resource attributes instead of detecting them again.

• OpenTelemetry Collector: On Kubernetes, add the k8s_attributes processor in passthrough mode and the resource_detection processor to the node-level Collector. On cloud Kubernetes, use the detector for your cloud provider. See the Kubernetes DaemonSet -> Gateway example in Configure hostname processors in the Collector.
• DDOT Collector: Keep the infraattributes processor in the DaemonSet configuration, as shown in the DDOT gateway installation guide.

If a gateway deployment reports the wrong host, see Gateway collector not forwarding host metadata.

Existing configurations using the Datadog Exporter

This applies to existing OpenTelemetry Collector configurations with the Datadog Exporter. For new Collector configurations, use the recommended OpenTelemetry Collector setup.

Run a Collector on each host or Kubernetes node, and configure host detection the same way as the recommended setup. The processor examples on this page work with the Datadog Exporter. For a complete Datadog Exporter example on Amazon EKS, see k8s-values.yaml.

If your Collector forwards to a gateway, also follow the gateway recommendations.

Diagnose hostname issues

Datadog emits the datadog.apm.hostname_issue gauge when an APM trace hostname is missing, resembles an ephemeral Kubernetes pod, or differs from the hostname reported by the Datadog Agent. The metric doesn’t affect billing.

A trace has at most one issue_type. Use the following table to find the likely cause and the fix:

Available metric tags include issue_type, host, env, service, version, and span_source. Affected spans receive the same issue_type tag, which you can use to find example traces and inspect their resource attributes.

The pod-like issue types use common Kubernetes pod naming patterns as a heuristic. After updating your configuration, inspect new traces to confirm that they no longer have the issue type.

Configure hostname processors in the Collector

Use the resource detection processor to detect host and cloud attributes, and the Kubernetes attributes processor to add Kubernetes metadata. Add the processors to your traces, metrics, and logs pipelines. On Kubernetes, list k8s_attributes before resource_detection. These examples work with both the OTLP HTTP exporter and the Datadog Exporter.

The recommended setup’s Host, Docker, and Kubernetes configurations already include these processors. Don’t replace them with these examples: the recommended k8s_attributes processor also sets service.name and other service attributes from pod metadata. Use these examples for gateways, existing Datadog Exporter configurations, and other Collector configurations without host detection.

The component names resource_detection and k8s_attributes require OpenTelemetry Collector Contrib v0.153.0 or later. Earlier versions use resourcedetection and k8sattributes. Later versions accept both names.

For complete, tested configurations, see the recommended OpenTelemetry Collector setup.

processors:
  resource_detection:
    detectors: [env, system]
    timeout: 2s
    override: true # Disable if incoming attributes (especially host.name) are verified correct
    system:
      resource_attributes:
        # Optional: send CPU and OS details to the Infrastructure Host List
        os.description:
          enabled: true
        host.arch:
          enabled: true
        host.cpu.vendor.id:
          enabled: true
        host.cpu.family:
          enabled: true
        host.cpu.model.id:
          enabled: true
        host.cpu.model.name:
          enabled: true
        host.cpu.stepping:
          enabled: true
        host.cpu.cache.l2.size:
          enabled: true

processors:
  resource_detection:
    detectors: [env]
    timeout: 2s
    override: true # Disable if incoming attributes (especially host.name) are verified correct

extraEnvs:
  # Non-cloud Kubernetes only
  - name: OTEL_RESOURCE_ATTRIBUTES
    value: "k8s.node.name=$(OTEL_K8S_NODE_NAME)"
serviceAccount:
  create: true
clusterRole:
  create: true
  rules:
    - apiGroups: [""]
      resources: ["pods", "namespaces", "nodes"]
      verbs: ["get", "watch", "list"]
    - apiGroups: ["apps"]
      resources: ["replicasets"]
      verbs: ["get", "list", "watch"]
    - apiGroups: ["extensions"]
      resources: ["replicasets"]
      verbs: ["get", "list", "watch"]

processors:
  k8s_attributes:
    passthrough: false
    auth_type: "serviceAccount"
    pod_association:
      - sources:
          - from: resource_attribute
            name: k8s.pod.uid
      - sources:
          - from: resource_attribute
            name: k8s.pod.ip
      - sources:
          - from: resource_attribute
            name: k8s.pod.name
          - from: resource_attribute
            name: k8s.namespace.name
      - sources:
          - from: connection
    extract:
      metadata:
        - k8s.pod.name
        - k8s.pod.uid
        - k8s.deployment.name
        - k8s.node.name
        - k8s.namespace.name
        - k8s.pod.start_time
        - k8s.replicaset.name
        - k8s.replicaset.uid
        - k8s.daemonset.name
        - k8s.daemonset.uid
        - k8s.job.name
        - k8s.job.uid
        - k8s.cronjob.name
        - k8s.statefulset.name
        - k8s.statefulset.uid
        - container.image.name
        - container.image.tag
        - container.id
        - k8s.container.name
      labels:
        - tag_name: kube_app_name
          key: app.kubernetes.io/name
          from: pod
        - tag_name: kube_app_instance
          key: app.kubernetes.io/instance
          from: pod
        - tag_name: kube_app_version
          key: app.kubernetes.io/version
          from: pod
        - tag_name: kube_app_component
          key: app.kubernetes.io/component
          from: pod
        - tag_name: kube_app_part_of
          key: app.kubernetes.io/part-of
          from: pod
        - tag_name: kube_app_managed_by
          key: app.kubernetes.io/managed-by
          from: pod
  resource_detection:
    detectors: [env, system]
    timeout: 2s
    override: true # Disable if incoming attributes (especially host.name) are verified correct
    system:
      resource_attributes:
        host.name:
          enabled: false # Containers report inaccurate host names

processors:
  k8s_attributes:
    passthrough: true
    auth_type: "serviceAccount"
  resource_detection:
    detectors: [env, system]
    timeout: 2s
    override: true # Disable if incoming attributes (especially host.name) are verified correct
    system:
      resource_attributes:
        host.name:
          enabled: false # Containers report inaccurate host names

processors:
  k8s_attributes:
    passthrough: false
    auth_type: "serviceAccount"
    pod_association:
      - sources:
          - from: resource_attribute
            name: k8s.pod.ip
    extract:
      metadata:
        - k8s.pod.name
        - k8s.pod.uid
        - k8s.deployment.name
        - k8s.node.name
        - k8s.namespace.name
        - k8s.pod.start_time
        - k8s.replicaset.name
        - k8s.replicaset.uid
        - k8s.daemonset.name
        - k8s.daemonset.uid
        - k8s.job.name
        - k8s.job.uid
        - k8s.cronjob.name
        - k8s.statefulset.name
        - k8s.statefulset.uid
        - container.image.name
        - container.image.tag
        - container.id
        - k8s.container.name
      labels:
        - tag_name: kube_app_name
          key: app.kubernetes.io/name
          from: pod
        - tag_name: kube_app_instance
          key: app.kubernetes.io/instance
          from: pod
        - tag_name: kube_app_version
          key: app.kubernetes.io/version
          from: pod
        - tag_name: kube_app_component
          key: app.kubernetes.io/component
          from: pod
        - tag_name: kube_app_part_of
          key: app.kubernetes.io/part-of
          from: pod
        - tag_name: kube_app_managed_by
          key: app.kubernetes.io/managed-by
          from: pod

processors:
  k8s_attributes:
    passthrough: false
    auth_type: "serviceAccount"
    pod_association:
      - sources:
          - from: resource_attribute
            name: k8s.pod.ip
    extract:
      metadata:
        - k8s.pod.name
        - k8s.pod.uid
        - k8s.deployment.name
        - k8s.node.name
        - k8s.namespace.name
        - k8s.pod.start_time
        - k8s.replicaset.name
        - k8s.replicaset.uid
        - k8s.daemonset.name
        - k8s.daemonset.uid
        - k8s.job.name
        - k8s.job.uid
        - k8s.cronjob.name
        - k8s.statefulset.name
        - k8s.statefulset.uid
        - container.image.name
        - container.image.tag
        - container.id
        - k8s.container.name
      labels:
        - tag_name: kube_app_name
          key: app.kubernetes.io/name
          from: pod
        - tag_name: kube_app_instance
          key: app.kubernetes.io/instance
          from: pod
        - tag_name: kube_app_version
          key: app.kubernetes.io/version
          from: pod
        - tag_name: kube_app_component
          key: app.kubernetes.io/component
          from: pod
        - tag_name: kube_app_part_of
          key: app.kubernetes.io/part-of
          from: pod
        - tag_name: kube_app_managed_by
          key: app.kubernetes.io/managed-by
          from: pod

Data collected

The following table lists resource attributes that these processors can add. The Datadog tag column shows the tag that Datadog creates from the attribute, where one exists.

Custom tagging

Custom host tags

As OTLP resource attributes

To add custom host tags, set resource attributes with the datadog.host.tag. prefix.

In an OpenTelemetry SDK, set them with the OTEL_RESOURCE_ATTRIBUTES environment variable:

OTEL_RESOURCE_ATTRIBUTES=datadog.host.tag.<CUSTOM_TAG_NAME>=<CUSTOM_TAG_VALUE>

In the Collector, set them with the resource processor:

processors:
  resource:
    attributes:
    - key: datadog.host.tag.<CUSTOM_TAG_NAME>
      action: upsert
      from_attribute: <SOURCE_ATTRIBUTE>

Note: With the Datadog Exporter, custom host tags require the opt-in described in Infrastructure List Host Information.

In the Datadog Exporter

For existing configurations with the Datadog Exporter, you can also set custom host tags in the exporter’s host_metadata section:

exporters:
  datadog:
    host_metadata:
      tags: ["team:infra", "<TAG_KEY>:<TAG_VALUE>"]

These tags apply only to telemetry whose hostname matches the hostname in the exporter’s host metadata. To tag telemetry from any host, use a processor. For all options, see the example Datadog Exporter configuration.

Host aliases

To set host aliases, set the datadog.host.aliases resource attribute. For example, use the transform processor:

processors:
  transform:
    trace_statements:
      - context: resource
        statements:
          - set(attributes["datadog.host.aliases"], ["alias1", "alias2", "alias3"])

Note: With the Datadog Exporter, host aliases require the opt-in described in Infrastructure List Host Information.

Custom container tags

To add custom container tags, set resource attributes with the datadog.container.tag. prefix.

In an OpenTelemetry SDK, set them with the OTEL_RESOURCE_ATTRIBUTES environment variable:

OTEL_RESOURCE_ATTRIBUTES=datadog.container.tag.<CUSTOM_TAG_NAME>=<CUSTOM_TAG_VALUE>

In the Collector, set them with the resource processor:

processors:
  resource:
    attributes:
    - key: datadog.container.tag.<CUSTOM_TAG_NAME>
      action: upsert
      from_attribute: <SOURCE_ATTRIBUTE>

Further reading

Additional helpful documentation, links, and articles:

Setting Up the OpenTelemetry CollectorDOCUMENTATION Mapping OpenTelemetry Semantic Conventions to HostnamesDOCUMENTATION