Dual shipping can impact billing if you are sending data to multiple Datadog organizations. For more information about the impact of this configuration, contact Datadog Support.
Overview
This document provides examples of Agent configurations for dual shipping different types of data (for example, APM, logs, Cluster Agent metrics, and so on) to multiple Datadog organizations.
Note: Use Observability Pipelines if you want to dual ship logs or split log traffic across different logging vendors, cloud storages, or SIEM providers.
For a full list of network traffic destinations, see Network Traffic.
Metrics and service checks
You can add the YAML configuration to your datadog.yaml or launch the Agent with the appropriate environment variables.
Note: Uploads to additional endpoints for the Continuous Profiler product are done through best-effort delivery.
The main endpoint has the highest priority. Uploads to additional endpoints are only handled after uploads to the main endpoint have completed successfully.
Responses from additional endpoints are not forwarded back to the profiler. Any errors during delivery to additional endpoints are logged in the Agent error logs.
To ensure autoscaling is resilient to failure, configure the Cluster Agent to run your metric queries for the HPA against your multiple Datadog regions with dual-shipped data. Configure the Datadog Cluster Agent manifest with several endpoints:
Use the Agent if you want to dual ship logs to multiple Datadog organizations. Use Observability Pipelines if you want to send logs to Datadog and external destinations.
TCP requires Agent version >= 6.6. HTTPS requires Agent version >= 6.13.
When setting up additional endpoints, you must explicitly set use_http to tell the Agent which transport to use. The same transport configuration is shared among all additional endpoints.
The is_reliable setting (First available in Agent 7.34.0) tells the Agent to treat this endpoint with the same priority as the primary endpoint. The primary endpoint is always reliable. This ensures that data is not missed if a destination becomes unavailable.
For example, if you’re sending data to the main endpoint and an additional endpoint with is_reliable: true, and one endpoint becomes unavailable, data continues to flow to the other endpoint. If both endpoints become unavailable, the Agent stops reading and sending data until at least one endpoint recovers. This ensures all data makes it to at least one reliable endpoint.
The is_reliable setting defaults to true in Agent 7.37.0+. Unreliable endpoints only send data if at least one reliable endpoint is available. You may define multiple additional endpoints with a mixed usage of is_reliable values. Datadog recommends that you use the default is_reliable setting.
You can add the YAML configuration to your datadog.yaml or launch the Agent with the appropriate environment variables.
When setting up additional endpoints, you must explicitly set use_http to tell the Agent which transport to use. The same transport configuration is shared among all additional endpoints.
The is_reliable setting (First available in Agent 7.34.0) tells the Agent to treat this endpoint with the same priority as the primary endpoint. The primary endpoint is always reliable. This ensures that data is not missed if a destination becomes unavailable.
For example, if you’re sending data to the main endpoint and an additional endpoint with is_reliable: true, and one endpoint becomes unavailable, data continues to flow to the other endpoint. If both endpoints become unavailable, the Agent stops reading and sending data until at least one endpoint recovers. This ensures all data makes it to at least one reliable endpoint.
The is_reliable setting defaults to true in Agent 7.37.0+. Unreliable endpoints only send data if at least one reliable endpoint is available. You may define multiple additional endpoints with a mixed usage of is_reliable values. Datadog recommends that you use the default is_reliable setting.
You can add the YAML configuration to your datadog.yaml or launch the Agent with the appropriate environment variables.
When setting up additional endpoints, you must explicitly set use_http to tell the Agent which transport to use. The same transport configuration is shared among all additional endpoints.
The is_reliable setting (First available in Agent 7.34.0) tells the Agent to treat this endpoint with the same priority as the primary endpoint. The primary endpoint is always reliable. This ensures that data is not missed if a destination becomes unavailable.
For example, if you’re sending data to the main endpoint and an additional endpoint with is_reliable: true, and one endpoint becomes unavailable, data continues to flow to the other endpoint. If both endpoints become unavailable, the Agent stops reading and sending data until at least one endpoint recovers. This ensures all data makes it to at least one reliable endpoint.
The is_reliable setting defaults to true in Agent 7.37.0+. Unreliable endpoints only send data if at least one reliable endpoint is available. You may define multiple additional endpoints with a mixed usage of is_reliable values. Datadog recommends that you use the default is_reliable setting.
You can add the YAML configuration to your datadog.yaml or launch the Agent with the appropriate environment variables.
When setting up additional endpoints, you must explicitly set use_http to tell the Agent which transport to use. The same transport configuration is shared among all additional endpoints.
The is_reliable setting (First available in Agent 7.34.0) tells the Agent to treat this endpoint with the same priority as the primary endpoint. The primary endpoint is always reliable. This ensures that data is not missed if a destination becomes unavailable.
For example, if you’re sending data to the main endpoint and an additional endpoint with is_reliable: true, and one endpoint becomes unavailable, data continues to flow to the other endpoint. If both endpoints become unavailable, the Agent stops reading and sending data until at least one endpoint recovers. This ensures all data makes it to at least one reliable endpoint.
The is_reliable setting defaults to true in Agent 7.37.0+. Unreliable endpoints only send data if at least one reliable endpoint is available. You may define multiple additional endpoints with a mixed usage of is_reliable values. Datadog recommends that you use the default is_reliable setting.
You can add the YAML configuration to your datadog.yaml or launch the Agent with the appropriate environment variables.
When setting up additional endpoints, you must explicitly set use_http to tell the Agent which transport to use. The same transport configuration is shared among all additional endpoints.
The is_reliable setting (First available in Agent 7.34.0) tells the Agent to treat this endpoint with the same priority as the primary endpoint. The primary endpoint is always reliable. This ensures that data is not missed if a destination becomes unavailable.
For example, if you’re sending data to the main endpoint and an additional endpoint with is_reliable: true, and one endpoint becomes unavailable, data continues to flow to the other endpoint. If both endpoints become unavailable, the Agent stops reading and sending data until at least one endpoint recovers. This ensures all data makes it to at least one reliable endpoint.
The is_reliable setting defaults to true in Agent 7.37.0+. Unreliable endpoints only send data if at least one reliable endpoint is available. You may define multiple additional endpoints with a mixed usage of is_reliable values. Datadog recommends that you use the default is_reliable setting.
You can add the YAML configuration to your datadog.yaml or launch the Agent with the appropriate environment variables.
When setting up additional endpoints, you must explicitly set use_http to tell the Agent which transport to use. The same transport configuration is shared among all additional endpoints.
The is_reliable setting (First available in Agent 7.34.0) tells the Agent to treat this endpoint with the same priority as the primary endpoint. The primary endpoint is always reliable. This ensures that data is not missed if a destination becomes unavailable.
For example, if you’re sending data to the main endpoint and an additional endpoint with is_reliable: true, and one endpoint becomes unavailable, data continues to flow to the other endpoint. If both endpoints become unavailable, the Agent stops reading and sending data until at least one endpoint recovers. This ensures all data makes it to at least one reliable endpoint.
The is_reliable setting defaults to true in Agent 7.37.0+. Unreliable endpoints only send data if at least one reliable endpoint is available. You may define multiple additional endpoints with a mixed usage of is_reliable values. Datadog recommends that you use the default is_reliable setting.
You can add the YAML configuration to your datadog.yaml or launch the Agent with the appropriate environment variables.
Dual shipping in Kubernetes
If you’re using the Datadog Agent Helm chart, you can configure these settings with a configmap. In the values.yaml, set useConfigMap: true
and add the relevant settings to customAgentConfig.
# agents.useConfigMap -- Configures a configmap to provide the agent configuration. Use this in combination with the `agents.customAgentConfig` parameter.useConfigMap:true# agents.customAgentConfig -- Specify custom contents for the datadog agent config (datadog.yaml)## ref: https://docs.datadoghq.com/agent/configuration/agent-configuration-files/?tab=agentv6## ref: https://github.com/DataDog/datadog-agent/blob/main/pkg/config/config_template.yaml## Note the `agents.useConfigMap` needs to be set to `true` for this parameter to be taken into account.customAgentConfig:additional_endpoints:"https://app.datadoghq.com":- apikey2- apikey3"https://app.datadoghq.eu":- apikey4logs_config:force_use_http:trueadditional_endpoints:- api_key:"apiKey2"Host:"agent-http-intake.logs.datadoghq.com"Port:443is_reliable:true
To avoid exposing your API key(s) in clear text inside the ConfigMap, you can also use the environment variable configuration and reference a Kubernetes secret. Here is an example to send metrics to an additional region:
Create a Kubernetes secret with your environment variable configuration value from this guide:
If you’re using the Datadog Agent operator, you can set the [key].customConfigurations.[key].configDataoverride key to set these settings. The example below replaces the datadog.yaml configuration file of the node Agent to send metrics and logs to additional regions.
To avoid exposing your API key(s) in clear text inside the ConfigMap, you can also use the environment variable configuration and reference a Kubernetes secret. Here is an example to send metrics to an additional region:
Create a Kubernetes secret with your environment variable configuration value from this guide:
