- Essentials
- In The App
- Infrastructure
- Application Performance
- Log Management
- Security Platform
- UX Monitoring
- Administration
This guide walks you through how to configure AWS PrivateLink for use with Datadog.
The overall process consists of configuring an internal endpoint in your VPC that local Datadog Agents can send data to. Your VPC endpoint is then peered with the endpoint within Datadog’s VPC.
Datadog exposes AWS PrivateLink endpoints in us-east-1.
However, to route traffic to Datadog’s PrivateLink offering in us-east-1
from other regions, use inter-region Amazon VPC peering. Inter-region VPC peering enables you to establish connections between VPCs across different AWS regions. This allows VPC resources in different regions to communicate with each other using private IP addresses. For more details, see Amazon VPC peering.
Connect the AWS Console to region us-east-1 and create a VPC endpoint.
Select Find service by name.
Fill the Service Name text box according to which service you want to establish AWS PrivateLink for:
Datadog | PrivateLink service name | Private DNS name |
---|---|---|
Logs (Agent HTTP intake) | com.amazonaws.vpce.us-east-1.vpce-svc-025a56b9187ac1f63 | agent-http-intake.logs.datadoghq.com |
Logs (User HTTP intake) | com.amazonaws.vpce.us-east-1.vpce-svc-0e36256cb6172439d | http-intake.logs.datadoghq.com |
API | com.amazonaws.vpce.us-east-1.vpce-svc-064ea718f8d0ead77 | api.datadoghq.com |
Metrics | com.amazonaws.vpce.us-east-1.vpce-svc-09a8006e245d1e7b8 | metrics.agent.datadoghq.com |
Containers | com.amazonaws.vpce.us-east-1.vpce-svc-0ad5fb9e71f85fe99 | orchestrator.datadoghq.com |
Process | com.amazonaws.vpce.us-east-1.vpce-svc-0ed1f789ac6b0bde1 | process.datadoghq.com |
Profiling | com.amazonaws.vpce.us-east-1.vpce-svc-022ae36a7b2472029 | intake.profile.datadoghq.com |
Traces | com.amazonaws.vpce.us-east-1.vpce-svc-0355bb1880dfa09c2 | trace.agent.datadoghq.com |
Click Verify. If this does not return Service name found, reach out to Datadog support.
Choose the VPC and subnets that should be peered with the Datadog VPC service endpoint.
Make sure that for Enable DNS name, Enable for this endpoint is checked:
Choose the security group of your choice to control what can send traffic to this VPC endpoint.
Note: The security group must accept inbound traffic on TCP port 443
.
Click Create endpoint at the bottom of the screen. If successful, the following is displayed:
Click on the VPC endpoint ID to check its status.
Wait for the status to move from Pending to Available. This can take up to 10 minutes. Once it shows Available, you can use AWS PrivateLink.
If you are collecting logs data, ensure your Agent is configured to send logs over HTTPS. If the data is not already there, add the following to the Agent datadog.yaml
configuration file:
logs_config:
use_http: true
If you are using the container Agent, set the following environment variable instead:
DD_LOGS_CONFIG_USE_HTTP=true
This configuration is required when sending logs to Datadog with AWS PrivateLink and the Datadog Agent, and is not required for the Lambda Extension. For more details, see Agent log collection.
If your Lambda Extension loads the Datadog API Key from AWS Secrets Manager using the ARN specified by the environment variable DD_API_KEY_SECRET_ARN
, you need to create a VPC endpoint for Secrets Manager.
Restart your Agent to send data to Datadog through AWS PrivateLink.
Datadog | PrivateLink service name |
---|---|
Metrics | com.amazonaws.vpce.us-east-1.vpce-svc-09a8006e245d1e7b8 |
Logs (Agent HTTP intake) | com.amazonaws.vpce.us-east-1.vpce-svc-025a56b9187ac1f63 |
Logs (User HTTP intake) | com.amazonaws.vpce.us-east-1.vpce-svc-0e36256cb6172439d |
API | com.amazonaws.vpce.us-east-1.vpce-svc-064ea718f8d0ead77 |
Process | com.amazonaws.vpce.us-east-1.vpce-svc-0ed1f789ac6b0bde1 |
Profiling | com.amazonaws.vpce.us-east-1.vpce-svc-022ae36a7b2472029 |
Traces | com.amazonaws.vpce.us-east-1.vpce-svc-0355bb1880dfa09c2 |
Containers | com.amazonaws.vpce.us-east-1.vpce-svc-0ad5fb9e71f85fe99 |
Click Verify. If this does not return Service name found, reach out to Datadog support.
Next, choose the VPC and subnets that should be peered with the Datadog VPC service endpoint. Do not select Enable DNS name as VPC peering requires DNS to be manually configured.
Choose the security group of your choice to control what can send traffic to this VPC endpoint.
Note: The security group must accept inbound traffic on TCP port 443
.
Click Create endpoint at the bottom of the screen. If successful, the following is displayed:
us-east-1
.Use the list below to map service and DNS name to different parts of Datadog:
Datadog | PrivateLink service name | Private DNS name |
---|---|---|
Metrics | com.amazonaws.vpce.us-east-1.vpce-svc-09a8006e245d1e7b8 | metrics.agent.datadoghq.com |
Logs (Agent HTTP intake) | com.amazonaws.vpce.us-east-1.vpce-svc-025a56b9187ac1f63 | agent-http-intake.logs.datadoghq.com |
Logs (User HTTP intake) | com.amazonaws.vpce.us-east-1.vpce-svc-0e36256cb6172439d | http-intake.logs.datadoghq.com |
API | com.amazonaws.vpce.us-east-1.vpce-svc-064ea718f8d0ead77 | api.datadoghq.com |
Process | com.amazonaws.vpce.us-east-1.vpce-svc-0ed1f789ac6b0bde1 | process.datadoghq.com |
Profiling | com.amazonaws.vpce.us-east-1.vpce-svc-022ae36a7b2472029 | intake.profile.datadoghq.com |
Traces | com.amazonaws.vpce.us-east-1.vpce-svc-0355bb1880dfa09c2 | trace.agent.datadoghq.com |
Containers | com.amazonaws.vpce.us-east-1.vpce-svc-0ad5fb9e71f85fe99 | orchestrator.datadoghq.com |
You can also find this information by interrogating the AWS API, DescribeVpcEndpointServices
, or by using the following CLI command: aws ec2 describe-vpc-endpoint-services --service-names <service-name>
.
For example, in the case of the Datadog metrics endpoint:
aws ec2 describe-vpc-endpoint-services --service-names com.amazonaws.vpce.us-east-1.vpce-svc-09a8006e245d1e7b8 | jq '.ServiceDetails[0].PrivateDnsName'
This returns metrics.agent.datadoghq.com
, the private hosted zone name that you need in order to associate with the VPC which the Agent traffic originates in. Overriding this record grabs all Metrics-related intake hostnames.
Within each new Route53 private hosted zone, create an A record with the same name. Toggle the Alias option, then under Route traffic to, choose Alias to VPC endpoint, us-east-1, and enter the DNS name of the VPC endpoint associated with the DNS name.
Note: To retrieve your DNS name, see the View endpoint service private DNS name configuration documentation.
Configure VPC peering and routing between the VPC in us-east-1
that contains the Datadog PrivateLink endpoints and the VPC in the region where the Datadog Agents run.
If the VPCs are in different AWS accounts, the VPC containing the Datadog Agent must be authorized to associate with the Route53 private hosted zones before continuing. Create a VPC association authorization for each Route53 private hosted zone using the region and VPC ID of the VPC where the Datadog Agent runs. This option is not available in the AWS Console. It must be configured using the AWS CLI, SDK, or API.
Edit the Route53 hosted zone to add the non-us-east-1 VPC.
VPCs that have the Private Hosted Zone (PHZ) attached need to have certain settings toggled on, specifically enableDnsHostnames
and enableDnsSupport
in the VPCs that the PHZ is associated with. See Considerations when working with a private hosted zone.
Restart the Agent to send data to Datadog through AWS PrivateLink.
The DNS names should resolve to IP addresses contained within the CIDR block of the VPC in us-east-1
, and connections to port 443
should succeed.
If DNS is resolving to public IP addresses, then the Route53 zone has not been associated with the VPC in the alternate region, or the A record does not exist.
If DNS resolves correctly, but connections to port 443
are failing, then VPC peering or routing may be misconfigured, or port 443 may not be allowed outbound to the CIDR block of the VPC in us-east-1
.
The VPCs with Private Hosted Zone (PHZ) attached need to have a couple of settings toggled on. Specifically, enableDnsHostnames
and enableDnsSupport
need to be turned on in the VPCs that the PHZ is associated with. See Amazon VPC settings.
If you are collecting logs data, ensure your Agent is configured to send logs over HTTPS. If the data is not already there, add the following to the Agent datadog.yaml
configuration file:
logs_config:
use_http: true
If you are using the container Agent, set the following environment variable instead:
DD_LOGS_CONFIG_USE_HTTP=true
This configuration is required when sending logs to Datadog with AWS PrivateLink and the Datadog Agent, and is not required for the Lambda Extension. For more details, see Agent log collection.
If your Lambda Extension loads the Datadog API Key from AWS Secrets Manager using the ARN specified by the environment variable DD_API_KEY_SECRET_ARN
, you need to create a VPC endpoint for Secrets Manager.
Additional helpful documentation, links, and articles: