Use the Container Agent to Tail Logs from the Host

Datadog recommends using STDOUT/STDERR to collect container logs.


Pods/containers have no access to host files by default, which also applies to the Agent. If you try to configure your container Agent to collect logs from host files, an error message similar to the one below appears:

    Type: file
    Path: /var/log/messages
    Status: Error: file /var/log/messages does not exist

To give the container Agent access to host files, mount the file or its directory to the container Agent. Review the list of Agent configuration files and directories for which host file and directory to mount based your OS.

Here are some examples for Kubernetes and Docker:

To mount the log files in your host to the Agent container, set the host log directory in the volumes section of your Agent manifest and the container log directory in volumeMounts section:

          - name: customlogs
            ## The desired log directory inside the agent container:
            mountPath: /container/var/test-dir/logs/

        - name: customlogs
            ## The directory in your host containing the log files.
            path: /var/test-dir/logs/

Next, configure the Agent to tail the files for log collection. To do this, you can mount a custom logs config into /conf.d/. The file name can be anything, as long as it has a .yaml extension.

It is preferable to use a ConfigMap to store configurations rather than mounting a host file directly. Here’s a sample ConfigMap manifest that has a logs.yaml file:

kind: ConfigMap
apiVersion: v1
     name: ddagent-logs-configmap
     namespace: default
     logs.yaml: |-
             - type: file
               service: syslog
               source: os
               ## Use the container log directory you set in the agent manifest
               path: /container/var/test-dir/logs/*.log

Create the ConfigMap object using the command:

kubectl create -f <configmap manifest>

Then, mount it under /conf.d/:

          - name: cm-logs
            mountPath: /conf.d/

        - name: cm-logs
            name: ddagent-logs-configmap

To mount the host log file, add a volume parameter in your Agent’s docker run command:

-v /<host log directory>/:<container log directory>/

Then, create a custom logs config locally:

  - type: file
    service: syslog
    source: os
    path: <container log path>/*.log

and mount it into /conf.d/. The file name can be anything:

-v <absolute path>/logs.yaml:/conf.d/logs.yaml

Your Agent’s Docker installation command should look like this:

docker run -d --name datadog-agent \
           --cgroupns host \
           --pid host \
           -e DD_API_KEY=<DATADOG_API_KEY> \
           -e DD_LOGS_ENABLED=true \
           -v /var/run/docker.sock:/var/run/docker.sock:ro \
           -v /proc/:/host/proc/:ro \
           -v /opt/datadog-agent/run:/opt/datadog-agent/run:rw \
           -v /sys/fs/cgroup/:/host/sys/fs/cgroup:ro \
           -v /<host log directory>/:<container log directory>/ \
           -v /<config location>/logs.yaml:/conf.d/logs.yaml \


After you have set this all up, you can deploy the Agent. You should be able to see something like the below when you run docker exec -it datadog-agent agent status:

Logs Agent

    Sending compressed logs in HTTPS to on port 443
    BytesSent: 10605
    EncodedBytesSent: 2144
    LogsProcessed: 32
    LogsSent: 31

    Type: file
    Path: /container/var/test-dir/logs/*.log
    Status: OK
      1 files tailed out of 1 files matching
    Inputs: /container/var/test-dir/logs/722bfb2cb35cc627-json.log