Kubernetes trace collection
Datadog の調査レポート: サーバーレスの状態 レポート: サーバーレスの状態

Kubernetes trace collection

このページは日本語には対応しておりません。随時翻訳に取り組んでいます。翻訳に関してご質問やご意見ございましたら、お気軽にご連絡ください。

In order to start collecting your application traces you must be running the Datadog Agent in your Kubernetes cluster.

Setup

To enable trace collection with your Agent, follow the instructions below:

  1. Configure the Datadog Agent to accept traces:

    • If you haven’t already, install the Helm chart.
    • Update your values.yaml file with the following APM configuration:

      datadog:
        ## @param apm - object - required
        ## Enable apm agent and provide custom configs
        #
        apm:
          ## @param enabled - boolean - optional - default: false
          ## Enable this to enable APM and tracing, on port 8126
          #
          enabled: true
      • Set your operating system. Add targetSystem: linux or targetSystem: windows to the top of your values.yaml.

      • Then, upgrade your Datadog Helm chart using the following command: helm upgrade -f values.yaml <RELEASE NAME> stable/datadog. Don’t forget to set the API key in the YAML file. If you did not set your operating system in values.yaml, add --set targetSystem=linux or --set targetSystem=windows to this command.

    To enable APM trace collection, open the DaemonSet configuration file and edit the following:

    • Allow incoming data from port 8126 (forwarding traffic from the host to the agent):

       # (...)
            ports:
              # (...)
              - containerPort: 8126
                hostPort: 8126
                name: traceport
                protocol: TCP
       # (...)
    • If using an old agent version (7.17 or lower), in addition to the steps above, set the DD_APM_NON_LOCAL_TRAFFIC and DD_APM_ENABLED variable to true in your env section of the datadog.yaml Agent manifest:

       # (...)
            env:
              # (...)
              - name: DD_APM_ENABLED
                value: 'true'
              - name: DD_APM_NON_LOCAL_TRAFFIC
                value: "true"
       # (...)
    Note: On minikube, you may receive an Unable to detect the kubelet URL automatically error. In this case, set DD_KUBELET_TLS_VERIFY=false.

  2. Configure your application pods to pull the host IP in order to communicate with the Datadog Agent: Use the downward API to pull the host IP; the application container needs the DD_AGENT_HOST environment variable that points to status.hostIP.

        apiVersion: apps/v1
        kind: Deployment
         # ...
            spec:
              containers:
              - name: "<CONTAINER_NAME>"
                image: "<CONTAINER_IMAGE>"/"<TAG>"
                env:
                  - name: DD_AGENT_HOST
                    valueFrom:
                      fieldRef:
                        fieldPath: status.hostIP
  3. Configure your application tracers to emit traces: Point your application-level tracers to where the Datadog Agent host is using the environment variable DD_AGENT_HOST. Refer to the language-specific APM instrumentation docs for more examples.

Agent Environment Variables

List of all environment variables available for tracing within the Agent running in Kubernetes:

Environment variableDescription
DD_API_KEYDatadog API Key
DD_PROXY_HTTPSSet up the URL for the proxy to use.
DD_APM_REPLACE_TAGSScrub sensitive data from your span’s tags.
DD_HOSTNAMEManually set the hostname to use for metrics if autodection fails, or when running the Datadog Cluster Agent.
DD_DOGSTATSD_PORTSet the DogStatsD port.
DD_APM_RECEIVER_SOCKETCollect your traces through a Unix Domain Sockets and takes priority over hostname and port configuration if set. Off by default, when set it must point to a valid sock file.
DD_BIND_HOSTSet the StatsD & receiver hostname.
DD_LOG_LEVELSet the logging level. (trace/debug/info/warn/error/critical/off)
DD_APM_ENABLEDWhen set to true, the Datadog Agent accepts trace metrics. Default value is true (Agent 7.18+)
DD_APM_CONNECTION_LIMITSets the maximum connection limit for a 30 second time window.
DD_APM_DD_URLDatadog API endpoint where traces are sent. For Datadog EU site set DD_APM_DD_URL to https://trace.agent.datadoghq.eu
DD_APM_RECEIVER_PORTPort that the Datadog Agent’s trace receiver listens on. Default value is 8126.
DD_APM_NON_LOCAL_TRAFFICAllow non-local traffic when tracing from other containers. Default value is true (Agent 7.18+)
DD_APM_IGNORE_RESOURCESConfigure resources for the Agent to ignore. Format should be comma separated, regular expressions. i.e. GET /ignore-me,(GET|POST) /and-also-me.
DD_APM_ANALYZED_SPANSConfigure the spans to analyze for transactions. Format should be comma separated instances of <SERVICE_NAME>|;<OPERATION_NAME>=1. i.e. my-express-app|;express.request=1,my-dotnet-app|;aspnet_core_mvc.request=1. You can also enable it automatically with the configuration parameter in the Tracing Client.
DD_APM_ENVSets the default environment for your traces.
DD_APM_MAX_EPSSets the maximum Analyzed Spans per second. Default is 200 events per second.
DD_APM_MAX_TPSSets the maximum traces per second. Default is 10 traces per second.

Further Reading