Configure Containers View

This page lists configuration options for the Containers page in Datadog. To learn more about the Containers page and its capabilities, see Containers View documentation.

Configuration options

Include or exclude containers

Include and exclude containers from real-time collection:

  • Exclude containers either by passing the environment variable DD_CONTAINER_EXCLUDE or by adding container_exclude: in your datadog.yaml main configuration file.
  • Include containers either by passing the environment variable DD_CONTAINER_INCLUDE or by adding container_include: in your datadog.yaml main configuration file.

Both arguments take an image name as value. Regular expressions are also supported.

For example, to exclude all Debian images except containers with a name starting with frontend, add these two configuration lines in your datadog.yaml file:

container_exclude: ["image:debian"]
container_include: ["name:frontend.*"]

Note: For Agent 5, instead of including the above in the datadog.conf main configuration file, explicitly add a datadog.yaml file to /etc/datadog-agent/, as the Process Agent requires all configuration options here. This configuration only excludes containers from real-time collection, not from Autodiscovery.

Scrubbing sensitive information

To prevent the leaking of sensitive data, you can scrub sensitive words in container YAML files. Container scrubbing is enabled by default for Helm charts, and some default sensitive words are provided:

  • password
  • passwd
  • mysql_pwd
  • access_token
  • auth_token
  • api_key
  • apikey
  • pwd
  • secret
  • credentials
  • stripetoken

You can set additional sensitive words by providing a list of words to the environment variable DD_ORCHESTRATOR_EXPLORER_CUSTOM_SENSITIVE_WORDS. This adds to, and does not overwrite, the default words.

Note: The additional sensitive words must be in lowercase, as the Agent compares the text with the pattern in lowercase. This means password scrubs MY_PASSWORD to MY_*******, while PASSWORD does not.

You need to setup this environment variable for the following agents:

  • process-agent
  • cluster-agent
env:
    - name: DD_ORCHESTRATOR_EXPLORER_CUSTOM_SENSITIVE_WORDS
      value: "customword1 customword2 customword3"

For example, because password is a sensitive word, the scrubber changes <MY_PASSWORD> in any of the following to a string of asterisks, ***********:

password <MY_PASSWORD>
password=<MY_PASSWORD>
password: <MY_PASSWORD>
password::::== <MY_PASSWORD>

However, the scrubber does not scrub paths that contain sensitive words. For example, it does not overwrite /etc/vaultd/secret/haproxy-crt.pem with /etc/vaultd/******/haproxy-crt.pem even though secret is a sensitive word.

Configure Orchestrator Explorer

Resource collection compatibility matrix

The following table presents the list of collected resources and the minimal Agent, Cluster Agent, and Helm chart versions for each.

ResourceMinimal Agent versionMinimal Cluster Agent version*Minimal Helm chart versionMinimal Kubernetes version
ClusterRoleBindings7.33.01.19.02.30.91.14.0
ClusterRoles7.33.01.19.02.30.91.14.0
Clusters7.33.01.18.02.10.01.17.0
CronJobs7.33.07.40.02.15.51.16.0
CustomResourceDefinitions7.51.07.51.03.39.2v1.16.0
CustomResources7.51.07.51.03.39.2v1.16.0
DaemonSets7.33.01.18.02.16.31.16.0
Deployments7.33.01.18.02.10.01.16.0
HorizontalPodAutoscalers7.33.07.51.02.10.01.1.1
Ingresses7.33.01.22.02.30.71.21.0
Jobs7.33.01.18.02.15.51.16.0
Namespaces7.33.07.41.02.30.91.17.0
Network Policies7.33.07.56.03.57.21.14.0
Nodes7.33.01.18.02.10.01.17.0
PersistentVolumeClaims7.33.01.18.02.30.41.17.0
PersistentVolumes7.33.01.18.02.30.41.17.0
Pods7.33.01.18.03.9.01.17.0
ReplicaSets7.33.01.18.02.10.01.16.0
RoleBindings7.33.01.19.02.30.91.14.0
Roles7.33.01.19.02.30.91.14.0
ServiceAccounts7.33.01.19.02.30.91.17.0
Services7.33.01.18.02.10.01.17.0
Statefulsets7.33.01.15.02.20.11.16.0
VerticalPodAutoscalers7.33.07.46.03.6.81.16.0

Note: After version 1.22, Cluster Agent version numbering follows Agent release numbering, starting with version 7.39.0.

Add custom tags to resources

You can add custom tags to Kubernetes resources to ease filtering inside the Kubernetes resources view.

Additional tags are added through the DD_ORCHESTRATOR_EXPLORER_EXTRA_TAGS environment variable.

Note: These tags only show up in the Kubernetes resources view.

Add the environment variable on both the Process Agent and the Cluster Agent by setting agents.containers.processAgent.env and clusterAgent.env in datadog-agent.yaml.

apiVersion: datadoghq.com/v2alpha1
kind: DatadogAgent
metadata:
  name: datadog
spec:
  global:
    credentials:
      apiKey: <DATADOG_API_KEY>
      appKey: <DATADOG_APP_KEY>
  features:
    liveContainerCollection:
      enabled: true
    orchestratorExplorer:
      enabled: true
  override:
    agents:
      containers:
        processAgent:
          env:
            - name: "DD_ORCHESTRATOR_EXPLORER_EXTRA_TAGS"
              value: "tag1:value1 tag2:value2"
    clusterAgent:
      env:
        - name: "DD_ORCHESTRATOR_EXPLORER_EXTRA_TAGS"
          value: "tag1:value1 tag2:value2"

Then, apply the new configuration:

kubectl apply -n $DD_NAMESPACE -f datadog-agent.yaml

If you are using the official Helm chart, add the environment variable on both the Process Agent and the Cluster Agent by setting agents.containers.processAgent.env and clusterAgent.env in values.yaml.

agents:
  containers:
    processAgent:
      env:
        - name: "DD_ORCHESTRATOR_EXPLORER_EXTRA_TAGS"
          value: "tag1:value1 tag2:value2"
clusterAgent:
  env:
    - name: "DD_ORCHESTRATOR_EXPLORER_EXTRA_TAGS"
      value: "tag1:value1 tag2:value2"

Then, upgrade your Helm chart.

Set the environment variable on both the Process Agent and Cluster Agent containers:

- name: DD_ORCHESTRATOR_EXPLORER_EXTRA_TAGS
  value: "tag1:value1 tag2:value2"

Collect custom resources

The Kubernetes Explorer automatically collects CustomResourceDefinitions (CRDs) by default.

Follow these steps to collect the custom resources that these CRDs define:

  1. In Datadog, open Kubernetes Explorer. On the left panel, under Select Resources, select Kubernetes > Custom Resources > Resource Definitions.

  2. Locate the CRD that defines the custom resource you want to visualize in the explorer. Under the Versions column, click the version tag you want to configure indexing for.

    A modal appears:

    The Collecting and Indexing modal. Contains two sections: Set up Datadog Agent, with copyable snippets for updating an Agent configuration, and Select indexed fields for filtering/sorting, with checkboxes for fields to index and a preview.

  3. Follow the instructions in the modal’s Set up Datadog Agent section to update the Agent configuration for clusters that are not collecting custom resources. The modal lists all such clusters, either because the Agent is not configured to collect custom resources, or because none are available in that cluster. If the Agent is configured and no custom resources exist, no action is required.

    1. Add the following configuration to datadog-values.yaml:

      datadog:
        #(...)
        orchestratorExplorer:
          customResources:
            - <CUSTOM_RESOURCE_NAME>
      
    2. Upgrade your Helm chart:

      helm upgrade -f datadog-values.yaml <RELEASE_NAME> datadog/datadog
      
    1. Install the Datadog Operator with an option that grants the Datadog Agent permission to collect custom resources:

      helm install datadog-operator datadog/datadog-operator --set clusterRole.allowReadAllResources=true
      
    2. Add the following configuration to your DatadogAgent manifest, datadog-agent.yaml:

      apiVersion: datadoghq.com/v2alpha1
      kind: DatadogAgent
      metadata:
        name: datadog
      spec:
        #(...)
        features:
          orchestratorExplorer:
            customResources:
              - <CUSTOM_RESOURCE_NAME>
      
    3. Apply your new configuration:

      kubectl apply -n $DD_NAMESPACE -f datadog-agent.yaml
      

    Each <CUSTOM_RESOURCE_NAME> must use the format group/version/kind.

  4. In the modal, under Select indexed fields for filtering/sorting, select the fields you want to index from the custom resource for filtering and sorting. For some CRDs, Datadog provides a default configuration. You can select additional fields if needed.

    After the Datadog Agent is set up, it collects available custom resources automatically. Indexing fields is optional.

    For arrays of objects, see the Indexing complex types section.

  5. Select Update Fields to save.

After the fields are indexed, you can add them as columns in the explorer and sort them, or include them in Saved Views. You can also filter on indexed fields using the prefix field#.

Indexing complex types

Indexing Configuration: A targets object[] array, with 'Group by' drop down options: no field, containerResource.container, containerResource.name, containerResource.value.type, etc.

For arrays of objects, two group-by strategies are available:

  • No field: Object’s nested fields are indexed solely on nested field name.
  • Field (for example: type, status, etc.): Object’s nested fields are indexed based on each unique field value.
Example: Filtering on DatadogPodAutoscaler custom resources

Consider these two custom resources:

Custom Resource 1 (CR1):

status:
    conditions:
        - type: HorizontalAbleToScale
          status: 'True'
        - type: VerticalAbleToApply
          status: 'False'

Custom Resource 2 (CR2):

status:
    conditions:
        - type: VerticalAbleToApply
          status: 'True'
        - type: HorizontalAbleToScale
          status: 'False'

You have the filtering possibilities on status.conditions based on the two indexing strategies:

Indexed fields for CR1:

status:
    conditions:
        type: [HorizontalAbleToScale, VerticalAbleToApply]
        status: ['True', 'False']

Indexed fields for CR2:

status:
    conditions:
        type: [VerticalAbleToApply, HorizontalAbleToScale]
        status: ['True', 'False']

Example queries:

Query 1:

field#status.conditions.status:"False"

Result: Returns CR1 and CR2. Both CRs have at least one object with status:"False"

Query 2:

field#status.conditions.status:"False" AND field#status.conditions.type:VerticalAbleToApply

Result: Returns CR1 and CR2. At least one status.condition object in each custom resource matches one of the filters—even if it’s not the same object that matches both filters.

Indexed fields for CR1:

status:
    conditions:
        - HorizontalAbleToScale:
              status: 'True'
        - VerticalAbleToApply:
              status: 'False'

Indexed fields for CR2:

status:
    conditions:
        - VerticalAbleToApply:
              status: 'True'
        - HorizontalAbleToScale:
              status: 'False'

Example query:

field#status.conditions.HorizontalAbleToScale.status:"False"

Result: Returns CR2. Only a status.condition object whose type:"HorizontalAbleToScale" and status:"False" is returned.

You can select up to 50 fields per resource. You can use the preview to validate your indexing choices.

Further reading