Kubernetes Tag Extraction
The Agent can create and assign tags to all metrics, traces, and logs emitted by a Pod, based on its labels or annotations.
If you are running the Agent as a binary on a host, configure your tag extractions with the Agent tab instructions. If you are running the Agent as a container in your Kubernetes cluster, configure your tag extraction with the Containerized Agent tab instructions.
The Agent can autodiscover and attach tags to all data emitted by the entire pods or an individual container within this pod. The list of tags attached automatically depends on the agent cardinality configuration.
Tag | Cardinality | Source | Requirement |
---|
container_id | High | Pod status | N/A |
display_container_name | High | Pod status | N/A |
pod_name | Orchestrator | Pod metadata | N/A |
oshift_deployment | Orchestrator | Pod annotation openshift.io/deployment.name | OpenShift environment and pod annotation must exist |
kube_ownerref_name | Orchestrator | Pod ownerref | Pod must have an owner |
kube_job | Orchestrator | Pod ownerref | Pod must be attached to a cronjob |
kube_job | Low | Pod ownerref | Pod must be attached to a job |
kube_replica_set | Low | Pod ownerref | Pod must be attached to a replica set |
kube_service | Low | Kubernetes service discovery | Pod is behind a Kubernetes service |
kube_daemon_set | Low | Pod ownerref | Pod must be attached to a DaemonSet |
kube_container_name | Low | Pod status | N/A |
kube_namespace | Low | Pod metadata | N/A |
kube_app_name | Low | Pod label app.kubernetes.io/name | Pod label must exist |
kube_app_instance | Low | Pod label app.kubernetes.io/instance | Pod label must exist |
kube_app_version | Low | Pod label app.kubernetes.io/version | Pod label must exist |
kube_app_component | Low | Pod label app.kubernetes.io/component | Pod label must exist |
kube_app_part_of | Low | Pod label app.kubernetes.io/part-of | Pod label must exist |
kube_app_managed_by | Low | Pod label app.kubernetes.io/managed-by | Pod label must exist |
env | Low | Pod label tags.datadoghq.com/env or container envvar DD_ENV | Unified service tagging enabled |
version | Low | Pod label tags.datadoghq.com/version or container envvar DD_VERSION | Unified service tagging enabled |
service | Low | Pod label tags.datadoghq.com/service or container envvar DD_SERVICE | Unified service tagging enabled |
pod_phase | Low | Pod status | N/A |
oshift_deployment_config | Low | Pod annotation openshift.io/deployment-config.name | OpenShift environment and pod annotation must exist |
kube_ownerref_kind | Low | Pod ownerref | Pod must have an owner |
kube_deployment | Low | Pod ownerref | Pod must be attached to a deployment |
kube_replication_controller | Low | Pod ownerref | Pod must be attached to a replication controller |
kube_stateful_set | Low | Pod ownerref | Pod must be attached to a statefulset |
persistentvolumeclaim | Low | Pod spec | A PVC must be attached to the pod |
kube_cronjob | Low | Pod ownerref | Pod must be attached to a cronjob |
image_name | Low | Pod spec | N/A |
short_image | Low | Pod spec | N/A |
image_tag | Low | Pod spec | N/A |
eks_fargate_node | Low | Pod spec | EKS Fargate environment |
Host tag
The Agent can attach Kubernetes environment information as “host tags”.
Tag | Cardinality | Source | Requirement |
---|
kube_cluster_name | Low | DD_CLUSTER_NAME envvar or cloud provider integration | DD_CLUSTER_NAME envvar or cloud provider integration enabled |
kube_node_role | Low | Node label node-role.kubernetes.io/<role> | Node label must exist |
Tag Autodiscovery
Starting with Agent v6.10+, the Agent can autodiscover tags from Pod annotations. It allows the Agent to associate tags to all data emitted by the entire pods or an individual container within this pod.
As a best practice in containerized environments, Datadog recommends using unified service tagging to help unify tags. Unified service tagging ties Datadog telemetry together through the use of three standard tags: env
, service
, and version
. To learn how to configure your environment with unified tagging, refer to the dedicated unified service tagging documentation.
To apply a <TAG_KEY>:<TAG_VALUE>
tag to all data emitted by a given pod and collected by the Agent use the following annotation on your pod:
annotations:
ad.datadoghq.com/tags: '{"<TAG_KEY>": "<TAG_VALUE>","<TAG_KEY_1>": "<TAG_VALUE_1>"}'
If you want to apply a <TAG_KEY>:<TAG_VALUE>
tag to an individual container <CONTAINER_IDENTIFIER>
within a pod, use the following annotation on your pod:
annotations:
ad.datadoghq.com/<CONTAINER_IDENTIFIER>.tags: '{"<TAG_KEY>": "<TAG_VALUE>","<TAG_KEY_1>": "<TAG_VALUE_1>"}'
Starting with Agent v7.17+, the Agent can Autodiscover tags from Docker labels. This process allows the Agent to associate custom tags to all data emitted by a container, without modifying the Agent datadog.yaml
file.
com.datadoghq.ad.tags: '["<TAG_KEY>:TAG_VALUE", "<TAG_KEY_1>:<TAG_VALUE_1>"]'
Starting with Agent v6.0+, the Agent can collect labels for a given node and use them as tags to attach to all metrics emitted by all pods on this node:
To extract a given node label <NODE_LABEL>
and transform it as a tag key <TAG_KEY>
within Datadog, add the following environment variable to the Datadog Agent:
DD_KUBERNETES_NODE_LABELS_AS_TAGS='{"<NODE_LABEL>": "<TAG_KEY>"}'
For example, you could set up:
DD_KUBERNETES_NODE_LABELS_AS_TAGS='{"app":"kube_app"}'
For Agent v7.24.0+, use the following environment variable configuration to add all node labels as tags to your metrics. In this example, the tags names are prefixed by <PREFIX>_
:
DD_KUBERNETES_NODE_LABELS_AS_TAGS='{"*":"<PREFIX>_%%label%%"}'
Note: Custom metrics may impact billing. See the custom metrics billing page for more information.
To extract a given node label <NODE_LABEL>
and transform it as a tag key <TAG_KEY>
within Datadog, add the following configuration block in the Agent datadog.yaml
configuration file:
kubernetes_node_labels_as_tags:
<NODE_LABEL>: <TAG_KEY>
For example, you could set up:
kubernetes_node_labels_as_tags:
app: kube_app
Starting with Agent v6.0+, the Agent can collect labels for a given pod and use them as tags to attach to all metrics emitted by this pod:
To extract a given pod label <POD_LABEL>
and transform it as a tag key <TAG_KEY>
within Datadog, add the following environment variable to the Datadog Agent:
DD_KUBERNETES_POD_LABELS_AS_TAGS='{"<POD_LABEL>": "<TAG_KEY>"}'
For example, you could set up:
DD_KUBERNETES_POD_LABELS_AS_TAGS='{"app":"kube_app"}'
For Agent v6.8.0+, use the following environment variable configuration to add all pod labels as tags to your metrics. In this example, the tags names are prefixed by <PREFIX>_
:
DD_KUBERNETES_POD_LABELS_AS_TAGS='{"*":"<PREFIX>_%%label%%"}'
Note: Custom metrics may impact billing. See the custom metrics billing page for more information.
To extract a given pod label <POD_LABEL>
and transform it as a Tag Key <TAG_KEY>
within Datadog, add the following configuration block in the Agent datadog.yaml
configuration file:
kubernetes_pod_labels_as_tags:
<POD_LABEL>: <TAG_KEY>
For example, you could set up:
kubernetes_pod_labels_as_tags:
app: kube_app
For Agent v6.8.0+, use the following environment variable configuration to add all pod labels as tags to your metrics. In this example, the tag names are prefixed by <PREFIX>_
:
kubernetes_pod_labels_as_tags:
*: <PREFIX>_%%label%%
Note: Custom metrics may impact billing. See the [custom metrics billing page][3] for more information.
Starting with Agent v6.0+, the Agent can collect annotations for a given pod and use them as tags to attach to all metrics emitted by this pod:
To extract a given pod label <POD_ANNOTATION>
and transform it as a tag key <TAG_KEY>
within Datadog, add the following environment variable to the Datadog Agent:
DD_KUBERNETES_POD_ANNOTATIONS_AS_TAGS='{"<POD_ANNOTATION>": "<TAG_KEY>"}'
For example, you could set up:
DD_KUBERNETES_POD_ANNOTATIONS_AS_TAGS='{"app":"kube_app"}'
For Agent v7.24.0+, use the following environment variable configuration to add all pod annotations as tags to your metrics. In this example, the tags names are prefixed by <PREFIX>_
:
DD_KUBERNETES_POD_ANNOTATIONS_AS_TAGS='{"*":"<PREFIX>_%%annotation%%"}'
Note: Custom metrics may impact billing. See the custom metrics billing page for more information.
To extract a given pod annotation <POD_ANNOTATION>
and transform it as a tag key <TAG_KEY>
within Datadog, add the following configuration block in the Agent datadog.yaml
configuration file:
kubernetes_pod_annotations_as_tags:
<POD_ANNOTATION>: <TAG_KEY>
For example, you could set up:
kubernetes_pod_annotations_as_tags:
app: kube_app
Starting with Agent v7.27+, the Agent can collect labels for a given namespace and use them as tags to attach to all metrics emitted by all pods in this namespace:
To extract a given namespace label <NAMESPACE_LABEL>
and transform it as a tag key <TAG_KEY>
within Datadog, add the following environment variable to the Datadog Agent:
DD_KUBERNETES_NAMESPACE_LABELS_AS_TAGS='{"<NAMESPACE_LABEL>": "<TAG_KEY>"}'
For example, you could set up:
DD_KUBERNETES_NAMESPACE_LABELS_AS_TAGS='{"app":"kube_app"}'
Use the following environment variable configuration to add all namespace labels as tags to your metrics. In this example, the tag names are prefixed by <PREFIX>_
:
DD_KUBERNETES_NAMESPACE_LABELS_AS_TAGS='{"*":"<PREFIX>_%%label%%"}'
Note: Custom metrics may impact billing. See the custom metrics billing page for more information.
To extract a given namespace label <NAMESPACE_LABEL>
and transform it as a tag key <TAG_KEY>
within Datadog, add the following configuration block in the Agent datadog.yaml
configuration file:
kubernetes_namespace_labels_as_tags:
<NAMESPACE_LABEL>: <TAG_KEY>
For example, you could set up:
kubernetes_namespace_labels_as_tags:
app: kube_app
Starting with Agent v7.32+, the Agent can collect container environment variables and use them as tags to attach to all metrics corresponding to the container. Both docker
and containerd
containers are supported:
To extract a given environment variable <ENV_VAR>
and transform it as a tag key <TAG_KEY>
within Datadog, add the following environment variable to the Datadog Agent:
DD_CONTAINER_ENV_AS_TAGS='{"<ENV_VAR>": "<TAG_KEY>"}'
For example:
DD_CONTAINER_ENV_AS_TAGS='{"app":"kube_app"}'
Note: Custom metrics may impact billing. See Custom Metrics Billing for more details.
To extract a given environment variable <ENV_VAR>
and transform it as a tag key <TAG_KEY>
within Datadog, add the following configuration block in the Agent datadog.yaml
configuration file:
container_env_as_tags:
<ENV_VAR>: <TAG_KEY>
For example:
container_env_as_tags:
app: kube_app
Starting with Agent v7.33+, the Agent can collect container labels and use them as tags. The agent attaches the tags to all metrics associated with the container.
The Agent can generate tags from container labels for both docker
and containerd
containers. In the case of containerd
, the minimum supported version is v1.5.6, because previous releases do not propagate labels correctly.
To extract a given container label <CONTAINER_LABEL>
and transform it to a tag key <TAG_KEY>
, add the following environment variable to the Datadog Agent:
DD_CONTAINER_LABELS_AS_TAGS='{"<CONTAINER_LABEL>":"<TAG_KEY>"}'
For example:
DD_CONTAINER_LABELS_AS_TAGS='{"app":"kube_app"}'
Note: Custom metrics may impact billing. See Custom Metrics Billing for more details.
To extract a given container label <CONTAINER_LABEL>
and transform it to a tag key <TAG_KEY>
, add the following configuration block in the Agent datadog.yaml
configuration file:
container_labels_as_tags:
<CONTAINER_LABEL>: <TAG_KEY>
For example:
container_labels_as_tags:
app: kube_app
Further Reading
Additional helpful documentation, links, and articles: