Kubernetes Tag Extraction

Kubernetes Tag Extraction

The Agent can create and assign tags to all metrics, traces, and logs emitted by a Pod, based on its labels or annotations.

If you are running the Agent as a binary on a host, configure your tag extractions with the Agent tab instructions. If you are running the Agent as a container in your Kubernetes cluster, configure your tag extraction with the Containerized Agent tab instructions.

Out-of-the-box tags

The Agent can autodiscover and attach tags to all data emitted by the entire pods or an individual container within this pod. The list of tags attached automatically depends on the agent cardinality configuration.

TagCardinalitySourceRequirement
container_idHighPod statusN/A
display_container_nameHighPod statusN/A
pod_nameOrchestratorPod metadataN/A
oshift_deploymentOrchestratorPod annotation openshift.io/deployment.nameOpenShift environment and pod annotation must exist
kube_ownerref_nameOrchestratorPod ownerrefPod must have an owner
kube_jobOrchestratorPod ownerrefPod must be attached to a job
kube_replica_setOrchestratorPod ownerrefPod must be attached to a replica set
kube_serviceOrchestratorKubernetes service discoveryPod is behind a Kubernetes service
kube_daemon_setLowPod ownerrefPod must be attached to a DaemonSet
kube_container_nameLowPod statusN/A
kube_namespaceLowPod metadataN/A
kube_app_nameLowPod label app.kubernetes.io/namePod label must exist
kube_app_instanceLowPod label app.kubernetes.io/instancePod label must exist
kube_app_versionLowPod label app.kubernetes.io/versionPod label must exist
kube_app_componentLowPod label app.kubernetes.io/componentPod label must exist
kube_app_part_ofLowPod label app.kubernetes.io/part-ofPod label must exist
kube_app_managed_byLowPod label app.kubernetes.io/managed-byPod label must exist
envLowPod label tags.datadoghq.com/env or container envvar DD_ENVUnified service tagging enabled
versionLowPod label tags.datadoghq.com/version or container envvar DD_VERSIONUnified service tagging enabled
serviceLowPod label tags.datadoghq.com/service or container envvar DD_SERVICEUnified service tagging enabled
pod_phaseLowPod statusN/A
oshift_deployment_configLowPod annotation openshift.io/deployment-config.nameOpenShift environment and pod annotation must exist
kube_ownerref_kindLowPod ownerrefPod must have an owner
kube_deploymentLowPod ownerrefPod must be attached to a deployment
kube_replication_controllerLowPod ownerrefPod must be attached to a replication controller
kube_stateful_setLowPod ownerrefPod must be attached to a statefulset
persistentvolumeclaimLowPod specA PVC must be attached to the pod
kube_cronjobLowPod ownerrefPod must be attached to a cronjob
image_nameLowPod specN/A
short_imageLowPod specN/A
image_tagLowPod specN/A

Host tag

The Agent can attach Kubernetes environment information as “host tags”.

TagCardinalitySourceRequirement
kube_cluster_nameLowDD_CLUSTER_NAME envvar or cloud provider integrationDD_CLUSTER_NAME envvar or cloud provider integration enabled
kube_node_roleLowNode label node-role.kubernetes.io/<role>Node label must exist

Tag Autodiscovery

Starting with Agent v6.10+, the Agent can autodiscover tags from Pod annotations. It allows the Agent to associate tags to all data emitted by the entire pods or an individual container within this pod.

As a best practice in containerized environments, Datadog recommends using unified service tagging to help unify tags. Unified service tagging ties Datadog telemetry together through the use of three standard tags: env, service, and version. To learn how to configure your environment with unified tagging, refer to the dedicated unified service tagging documentation.

To apply a <TAG_KEY>:<TAG_VALUE> tag to all data emitted by a given pod and collected by the Agent use the following annotation on your pod:

annotations:
  ad.datadoghq.com/tags: '{"<TAG_KEY>": "<TAG_VALUE>","<TAG_KEY_1>": "<TAG_VALUE_1>"}'

If you want to apply a <TAG_KEY>:<TAG_VALUE> tag to an individual container <CONTAINER_IDENTIFIER> within a pod, use the following annotation on your pod:

annotations:
  ad.datadoghq.com/<CONTAINER_IDENTIFIER>.tags: '{"<TAG_KEY>": "<TAG_VALUE>","<TAG_KEY_1>": "<TAG_VALUE_1>"}'

Starting with Agent v7.17+, the Agent can Autodiscover tags from Docker labels. This process allows the Agent to associate custom tags to all data emitted by a container, without modifying the Agent datadog.yaml file.

com.datadoghq.ad.tags: '["<TAG_KEY>:TAG_VALUE", "<TAG_KEY_1>:<TAG_VALUE_1>"]'

Node labels as tags

Starting with Agent v6.0+, the Agent can collect labels for a given node and use them as tags to attach to all metrics emitted by all pods on this node:

To extract a given node label <NODE_LABEL> and transform it as a tag key <TAG_KEY> within Datadog, add the following environment variable to the Datadog Agent:

DD_KUBERNETES_NODE_LABELS_AS_TAGS='{"<NODE_LABEL>": "<TAG_KEY>"}'

For example, you could set up:

DD_KUBERNETES_NODE_LABELS_AS_TAGS='{"app":"kube_app"}'

For Agent v7.24.0+, use the following environment variable configuration to add all node labels as tags to your metrics. In this example, the tags names are prefixed by <PREFIX>_:

DD_KUBERNETES_NODE_LABELS_AS_TAGS='{"*":"<PREFIX>_%%label%%"}'

Note: Custom metrics may impact billing. See the custom metrics billing page for more information.

To extract a given node label <NODE_LABEL> and transform it as a tag key <TAG_KEY> within Datadog, add the following configuration block in the Agent datadog.yaml configuration file:

kubernetes_node_labels_as_tags:
  <NODE_LABEL>: <TAG_KEY>

For example, you could set up:

kubernetes_node_labels_as_tags:
  app: kube_app

Pod labels as tags

Starting with Agent v6.0+, the Agent can collect labels for a given pod and use them as tags to attach to all metrics emitted by this pod:

To extract a given pod label <POD_LABEL> and transform it as a tag key <TAG_KEY> within Datadog, add the following environment variable to the Datadog Agent:

DD_KUBERNETES_POD_LABELS_AS_TAGS='{"<POD_LABEL>": "<TAG_KEY>"}'

For example, you could set up:

DD_KUBERNETES_POD_LABELS_AS_TAGS='{"app":"kube_app"}'

For Agent v6.8.0+, use the following environment variable configuration to add all pod labels as tags to your metrics. In this example, the tags names are prefixed by <PREFIX>_:

DD_KUBERNETES_POD_LABELS_AS_TAGS='{"*":"<PREFIX>_%%label%%"}'

Note: Custom metrics may impact billing. See the custom metrics billing page for more information.

To extract a given pod label <POD_LABEL> and transform it as a Tag Key <TAG_KEY> within Datadog, add the following configuration block in the Agent datadog.yaml configuration file:

kubernetes_pod_labels_as_tags:
  <POD_LABEL>: <TAG_KEY>

For example, you could set up:

kubernetes_pod_labels_as_tags:
  app: kube_app

For Agent v6.8.0+, use the following environment variable configuration to add all pod labels as tags to your metrics. In this example, the tag names are prefixed by <PREFIX>_:

kubernetes_pod_labels_as_tags:
  *: <PREFIX>_%%label%%

Note: Custom metrics may impact billing. See the [custom metrics billing page][3] for more information.

Pod annotations as tags

Starting with Agent v6.0+, the Agent can collect annotations for a given pod and use them as tags to attach to all metrics emitted by this pod:

To extract a given pod label <POD_ANNOTATION> and transform it as a tag key <TAG_KEY> within Datadog, add the following environment variable to the Datadog Agent:

DD_KUBERNETES_POD_ANNOTATIONS_AS_TAGS='{"<POD_ANNOTATION>": "<TAG_KEY>"}'

For example, you could set up:

DD_KUBERNETES_POD_ANNOTATIONS_AS_TAGS='{"app":"kube_app"}'

For Agent v7.24.0+, use the following environment variable configuration to add all pod annotations as tags to your metrics. In this example, the tags names are prefixed by <PREFIX>_:

DD_KUBERNETES_POD_ANNOTATIONS_AS_TAGS='{"*":"<PREFIX>_%%annotation%%"}'

Note: Custom metrics may impact billing. See the custom metrics billing page for more information.

To extract a given pod annotation <POD_ANNOTATION> and transform it as a tag key <TAG_KEY> within Datadog, add the following configuration block in the Agent datadog.yaml configuration file:

kubernetes_pod_annotations_as_tags:
  <POD_ANNOTATION>: <TAG_KEY>

For example, you could set up:

kubernetes_pod_annotations_as_tags:
  app: kube_app

Namespace labels as tags

Starting with Agent v7.27+, the Agent can collect labels for a given namespace and use them as tags to attach to all metrics emitted by all pods in this namespace:

To extract a given namespace label <NAMESPACE_LABEL> and transform it as a tag key <TAG_KEY> within Datadog, add the following environment variable to the Datadog Agent:

DD_KUBERNETES_NAMESPACE_LABELS_AS_TAGS='{"<NAMESPACE_LABEL>": "<TAG_KEY>"}'

For example, you could set up:

DD_KUBERNETES_NAMESPACE_LABELS_AS_TAGS='{"app":"kube_app"}'

Use the following environment variable configuration to add all namespace labels as tags to your metrics. In this example, the tag names are prefixed by <PREFIX>_:

DD_KUBERNETES_NAMESPACE_LABELS_AS_TAGS='{"*":"<PREFIX>_%%label%%"}'

Note: Custom metrics may impact billing. See the custom metrics billing page for more information.

To extract a given namespace label <NAMESPACE_LABEL> and transform it as a tag key <TAG_KEY> within Datadog, add the following configuration block in the Agent datadog.yaml configuration file:

kubernetes_namespace_labels_as_tags:
  <NAMESPACE_LABEL>: <TAG_KEY>

For example, you could set up:

kubernetes_namespace_labels_as_tags:
  app: kube_app

Further Reading