Overview

Setting up Network Path involves configuring your environment to monitor and trace the network routes between your services and endpoints. This helps identify bottlenecks, latency issues, and potential points of failure in your network infrastructure. Network Path allows you to manually configure individual network paths or automatically discover them, depending on your needs.

Note: If your network configuration restricts outbound traffic, follow the setup instructions on the Agent proxy configuration documentation.

Setup

Monitor individual paths

You can monitor specific network paths by defining them in the Agent configuration file located at /etc/datadog-agent/conf.d/network_path.d/conf.yaml.

To get started, copy the example configuration, remove the .example extension, and update it with your desired settings, or use one of the environment-specific configurations below. For performance optimization, see increase the number of workers.

Agent v7.59+ is required.

  1. Enable the system-probe traceroute module in /etc/datadog-agent/system-probe.yaml by adding the following:

    traceroute:
      enabled: true
    
  2. Enable network_path to monitor new destinations from this Agent by creating or editing the /etc/datadog-agent/conf.d/network_path.d/conf.yaml file:

    init_config:
      min_collection_interval: 60 # in seconds, default 60 seconds
    instances:
      # configure the endpoints you want to monitor, one check instance per endpoint
      # warning: Do not set the port when using UDP. Setting the port when using UDP can cause traceroute calls to fail and falsely report an unreachable destination.
    
      - hostname: api.datadoghq.eu # endpoint hostname or IP
        protocol: TCP
        port: 443
        tags:
          - "tag_key:tag_value"
          - "tag_key2:tag_value2"
        min_collection_interval: 120 # set min_collection_interval at the instance level
      ## optional configs:
      # max_ttl: 30 # max traderoute TTL, default is 30
      # timeout: 1000 # timeout in milliseconds per hop, default is 1s
    
      # more endpoints
      - hostname: 1.1.1.1 # endpoint hostname or IP
        protocol: UDP
        tags:
          - "tag_key:tag_value"
          - "tag_key2:tag_value2"
    
  3. Restart the Agent after making these configuration changes to start seeing network paths.

Agent v7.61+ is required.

Note: Windows only supports TCP traceroutes.

  1. Enable the system-probe traceroute module in %ProgramData%\Datadog\system-probe.yaml by adding the following:

    traceroute:
      enabled: true
    
  2. Enable network_path to monitor new destinations from this Agent by creating or editing the %ProgramData%\Datadog\conf.d\network_path.d\conf.yaml file:

    init_config:
      min_collection_interval: 60 # in seconds, default 60 seconds
    instances:
      # configure the endpoints you want to monitor, one check instance per endpoint
      # warning: Do not set the port when using UDP. Setting the port when using UDP can cause traceroute calls to fail and falsely report an unreachable destination.
    
      - hostname: api.datadoghq.eu # endpoint hostname or IP
        protocol: TCP
        port: 443
        tags:
          - "tag_key:tag_value"
          - "tag_key2:tag_value2"
        min_collection_interval: 120 # set min_collection_interval at the instance level
      ## optional configs:
      # max_ttl: 30 # max traderoute TTL, default is 30
      # timeout: 1000 # timeout in milliseconds per hop, default is 1s
    
      # more endpoints
      - hostname: 1.1.1.1 # endpoint hostname or IP
        protocol: TCP
        tags:
          - "tag_key:tag_value"
          - "tag_key2:tag_value2"
    
  3. Restart the Agent after making these configuration changes to start seeing network paths.

Limitations:

  • In Windows environments, the Agent uses UDP by default to monitor individual paths. If the protocol is not specified in the configuration, the Agent attempts a UDP traceroute, and any errors are logged. To work around this, ensure the protocol is set to TCP. For example:

    init_config:
      min_collection_interval: 60 # in seconds, default 60 seconds
    instances:
      - hostname: api.datadoghq.eu # endpoint hostname or IP
        protocol: TCP
        port: 443 # optional port number, default is 80
    
  • In Windows Client OS environments, raw packets are not supported. To work around this, set protocol: TCP and tcp_method: syn_socket. Agent v7.67+ and Windows version 2004 (10.0; Build 19041) or later are required. For example:

    init_config:
      min_collection_interval: 60 # in seconds, default 60 seconds
    instances:
      - hostname: api.datadoghq.eu # endpoint hostname or IP
        protocol: TCP
        port: 443 # optional port number, default is 80
        tcp_method: syn_socket
    

Agent v7.59+ is required.

Helm chart v3.109.1+ is required. For more information, reference the Datadog Helm Chart documentation and the documentation for Kubernetes and Integrations.

To enable Network Path with Kubernetes using Helm, add the following to your values.yaml file.

datadog:
  traceroute:
    enabled: true
  confd:
    network_path.yaml: |-
      init_config:
        min_collection_interval: 60 # in seconds, default 60 seconds
      instances:
        # configure the endpoints you want to monitor, one check instance per endpoint
        # warning: Do not set the port when using UDP. Setting the port when using UDP can cause traceroute calls to fail and falsely report an unreachable destination.

        - hostname: api.datadoghq.eu # endpoint hostname or IP
          protocol: TCP
          port: 443
          tags:
            - "tag_key:tag_value"
            - "tag_key2:tag_value2"
          min_collection_interval: 120 # set min_collection_interval at the instance level
        ## optional configs:
        # max_ttl: 30 # max traderoute TTL, default is 30
        # timeout: 1000 # timeout in milliseconds per hop, default is 1s

        # more endpoints
        - hostname: 1.1.1.1 # endpoint hostname or IP
          protocol: UDP
          tags:
            - "tag_key:tag_value"
            - "tag_key2:tag_value2"

Datadog Autodiscovery allows you to enable Network Path on a per-service basis through Kubernetes annotations.

Helm chart v3.109.1+ is required. For more information, see the Datadog Helm Chart documentation.
  1. Enable the traceroute module in the Datadog values.yaml file, which the Network Path integration depends on.

    datadog:
      traceroute:
        enabled: true
    
  2. After the module is enabled, Datadog automatically detects Network Path annotations added to your Kubernetes pod. For more information, see Kubernetes and Integrations.

    apiVersion: v1
    kind: Pod
    # (...)
    metadata:
      name: '<POD_NAME>'
      annotations:
        ad.datadoghq.com/<CONTAINER_NAME>.checks: |
          {
            "network_path": {
              "init_config": {
                "min_collection_interval": 300
              },
              "instances": [
                    {
                      "protocol": "TCP",
                      "port": 443,
                      "source_service": "<CONTAINER_NAME>",
                      "tags": [
                        "tag_key:tag_value",
                        "tag_key2:tag_value2"
                      ],
                      "hostname": "api.datadoghq.eu"
                    },
                    {
                      "protocol": "UDP",
                      "source_service": "<CONTAINER_NAME>",
                      "tags": [
                        "tag_key:tag_value",
                        "tag_key2:tag_value2"
                      ],
                      "hostname": "1.1.1.1"
                    },
              ]
            }
          }
        # (...)
    spec:
      containers:
        - name: '<CONTAINER_NAME>'
    # (...)
    

    If you define pods indirectly (with deployments, ReplicaSets, or ReplicationControllers), add pod annotations under spec.template.metadata.

Increase the number of workers

Network Path monitoring for individual paths runs as an Agent Integration. The number of concurrent workers is controlled by the check_runners setting in the datadog.yaml file.

To increase the number of workers, add the following configuration to your datadog.yaml file:

## @param check_runners - integer - optional - default: 4
## @env DD_CHECK_RUNNERS - integer - optional - default: 4
## The `check_runners` refers to the number of concurrent check runners available for check instance execution.
## The scheduler attempts to spread the instances over the collection interval and will _at most_ be
## running the number of check runners instances concurrently.
##
## The level of concurrency has effects on the Agent's: RSS memory, CPU load, resource contention overhead, etc.
#
check_runners: <NUMBER_OF_WORKERS>

Network traffic paths (experimental)

Prerequisites: CNM must be enabled.

Note: Network traffic paths is experimental and is not yet stable. Do not deploy network traffic paths widely in a production environment.

Configure network traffic paths to allow the Agent to automatically discover and monitor network paths based on actual network traffic, eliminating the need to manually configure individual endpoints. See exclude CIDR ranges to filter specific network ranges.

Enabling Network Path to automatically detect paths can generate a significant number of logs, particularly when monitoring network paths across a large number of hosts.

Agent v7.59+ is required.

  1. Enable the system-probe traceroute module in /etc/datadog-agent/system-probe.yaml by adding the following:

    traceroute:
      enabled: true
    
  2. Enable network_path to monitor CNM connections by creating or editing the /etc/datadog-agent/datadog.yaml file:

    network_path:
      connections_monitoring:
        enabled: true
      # collector:
        # workers: <NUMBER OF WORKERS> # default 4
    

    For full configuration details, reference the example config, or use the following:

    network_path:
      connections_monitoring:
        ## @param enabled - bool - required - default:false
        ## Enable network path collection
        #
        enabled: true
      collector:
        ## @param workers - int - optional - default:4
        ## Number of workers that can collect paths in parallel
        ## Recommendation: leave at default
        #
        # workers: <NUMBER OF WORKERS> # default 4
    
        #@env DD_NETWORK_PATH_COLLECTOR_PATHTEST_INTERVAL - integer - optional - default: 10m
        # The `pathtest_interval` refers to the traceroute run interval for monitored connections.
        # pathtest_interval: 10m
    
        # @param pathtest_ttl - integer - optional - default: 35m
        # @env DD_NETWORK_PATH_COLLECTOR_PATHTEST_TTL - integer - optional - default: 35m
        # The `pathtest_ttl` refers to the duration (time-to-live) a connection will be monitored when it's not seen anymore.
        # The TTL is reset each time the connection is seen again.
        # pathtest_ttl: 35m
    
  3. Restart the Agent after making these configuration changes to start seeing network paths.

Agent v7.61+ is required.

  1. Enable the system-probe traceroute module in %ProgramData%\Datadog\system-probe.yaml by adding the following:

    traceroute:
      enabled: true
    
  2. Enable network_path to monitor CNM connections by creating or editing the %ProgramData%\Datadog\datadog.yaml file:

    network_path:
      connections_monitoring:
        enabled: true
      # collector:
        # workers: <NUMBER OF WORKERS> # default 4
    

    For full configuration details, reference the example config, or use the following:

    network_path:
      connections_monitoring:
        ## @param enabled - bool - required - default:false
        ## Enable network path collection
        #
        enabled: true
      collector:
        ## @param workers - int - optional - default:4
        ## Number of workers that can collect paths in parallel
        ## Recommendation: leave at default
        #
        # workers: <NUMBER OF WORKERS> # default 4
    
        #@env DD_NETWORK_PATH_COLLECTOR_PATHTEST_INTERVAL - integer - optional - default: 10m
        # The `pathtest_interval` refers to the traceroute run interval for monitored connections.
        # pathtest_interval: 10m
    
        # @param pathtest_ttl - integer - optional - default: 35m
        # @env DD_NETWORK_PATH_COLLECTOR_PATHTEST_TTL - integer - optional - default: 35m
        # The `pathtest_ttl` refers to the duration (time-to-live) a connection will be monitored when it's not seen anymore.
        # The TTL is reset each time the connection is seen again.
        # pathtest_ttl: 35m
    
  3. Restart the Agent after making these configuration changes to start seeing network paths.

Agent v7.59+ is required.

To enable Network Path with Kubernetes using Helm, add the following to your values.yaml file. Note: Helm chart v3.124.0+ is required. For more information, reference the Datadog Helm Chart documentation and the documentation for Kubernetes and Integrations.

datadog:
  networkPath:
    connectionsMonitoring:
      enabled: true
  ## Set to true to enable the Traceroute Module of the System Probe
  traceroute:
    enabled: true

  ## @param collector - custom object - optional
  ## Configuration related to Network Path Collector.
  #
  collector:
    ## @param workers - integer - optional - default: 4
    ## @env DD_WORKERS - integer - optional - default: 4
    ## The `workers` refers to the number of concurrent workers available for network path execution.
    #
    # workers: 4
    #@env DD_NETWORK_PATH_COLLECTOR_PATHTEST_INTERVAL - integer - optional - default: 10m
    # The `pathtest_interval` refers to the traceroute run interval for monitored connections.
    # pathtest_interval: 10m

    # @param pathtest_ttl - integer - optional - default: 35m
    # @env DD_NETWORK_PATH_COLLECTOR_PATHTEST_TTL - integer - optional - default: 35m
    # The `pathtest_ttl` refers to the duration (time-to-live) a connection will be monitored when it's not seen anymore.
    # The TTL is reset each time the connection is seen again.
    # pathtest_ttl: 35m

Exclude CIDR ranges

Classless Inter-Domain Routing (CIDR) ranges define blocks of IP addresses using network prefixes. You may want to exclude certain CIDR ranges from network traffic paths to:

  • Reduce monitoring overhead for internal networks
  • Focus on external traffic patterns
  • Exclude known infrastructure ranges that don’t require monitoring

To exclude specific CIDR ranges from network traffic paths, configure the following in your /etc/datadog-agent/datadog.yaml file:

network_path:
    connections_monitoring:
        enabled: true # enable network path collection
    collector:
        source_excludes:
            "10.0.0.0/8":
                - "*" # all ports
        dest_excludes:
            "10.0.0.0/8":
                - "*" # all ports
            "8.8.8.8":
                - "53" # dns
                - "33434-33464" # traceroute range

Troubleshooting

Use the following guidelines to troubleshoot issues with Network Path. If you need additional help, contact Datadog Support.

No Network Path data in the UI

If no data appears in the Network Path UI, the feature may not be fully enabled. Network Path requires the following:

  1. The traceroute module must be enabled in your system-probe.yaml file:

    traceroute:
      enabled: true
    
  2. At least one Network Path feature must be active, such as:

Error: status code: 404

If you encounter an error like the following:

Error: failed to trace path: traceroute request failed: Probe Path <path>, url: <url>, status code: 404
  • This indicates that the traceroute module is not enabled. Ensure the traceroute module is enabled in your system-probe.yaml file.

Further Reading