Overview

Datadog adaptive sampling helps you capture more relevant traces while remaining close to a specific budget (ingested gigabytes).

When you choose adaptive sampling as your sampling strategy, you select a target monthly volume for trace ingestion for one or more services. This ensures that the consumption of these services matches the target volume at the end of the month, while keeping visibility over their endpoints.

Adaptive sampling uses remote configuration plus the existing sampling rules mechanisms to dynamically adjust sampling rates for each environment, service, and resource combination. This helps you to:

• Match your specified monthly budget.
• Ensure visibility for low-traffic services and endpoints by capturing at least one trace for each combination of service, resource, and environment every 5 minutes.

To configure services to use adaptive sampling, follow the instructions listed below.

Requirements

• Datadog Agent 7.53.0 or higher.
• Remote Configuration enabled for your Agent.
• APM Remote Configuration Write permission.
  Note: If you don’t have this permission, ask your Datadog admin to update your permissions from your organization settings.

Tracing library versions

The following table lists minimum tracing library versions required for adaptive sampling:

Configure the adaptive sampling target

To get started with adaptive sampling, you first need to pick a target strategy setting:

• Set Budget by Number of APM Hosts: Configure a budget that is proportional to your allotment and the number of services onboarded (for example, based on the number of APM hosts)
• Set Budget by Data Volume: Configure a fixed target in gigabytes per month

To set the adaptive sampling monthly target:

1. Navigate to the Ingestion Control page.
2. Click Manage Adaptive Sampling Target.

3. Choose a target strategy for sampling:
   • Set budget by number of APM hosts
   • Set budget by data volume
4. Click Apply.

Set budget by number of APM hosts (Recommended)

Set your monthly target to a percentage of your allotment. At the bottom of the page, you are provided with a more complete explanation of how that percentage is converted in a monthly target volume. It is the product of:

• The global allotment: 150GB * number_of_APM_hosts + 50GB * number_of_traced_serverless_invocations (if applicable) + 10GB * number_of_fargate_tasks (if applicable)
• The percentage of allotment configured above
• The contribution of onboarded services to the allotment. For example, if the services onboarded to adaptive sampling contribute to 10% of the total ingested volume, Datadog targets 10% of the global allotment. This number increases with the number of services onboarded.

That monthly target volume is recomputed every 30 minutes.

Set budget by data volume

If you are configuring the first service to adaptive sampling, ensure that the ingestion volume target is >0. For subsequent services, you should increase the allocated budget after the new service is onboarded to account for the new volume.

Configure adaptive sampling for a service

View sampling rates by resource for a service

Before you configure adaptive sampling for a service, you can view the current ingestion configuration for the service.

To see configured sampling rates:

1. Navigate to the Ingestion Control page.
2. Click a service to view the Service Ingestion Summary.
3. View the table listing the applied sampling rates by resource of the service.

The table includes:

• Ingested bytes: Ingested bytes from spans of the service and resource.
• Downstream bytes: Ingested bytes from spans where the sampling decision starts from that service and resource, including downstream services.
• Configuration: Source of the resource sampling rate:
  • AUTOMATIC: Default head-based sampling mechanism from the Agent.
  • CONFIGURED LOCAL: Sampling rule set locally in the tracing library.
  • CONFIGURED REMOTE: Remote sampling rule set from the Datadog UI.
  • ADAPTIVE REMOTE: Adaptive sampling rules set by Datadog.

Onboard a service to adaptive sampling

To onboard a service to adaptive sampling:

1. Navigate to the Ingestion Control page.
2. Click a service to view the Service Ingestion Summary.
3. Click Manage Ingestion Rate.
4. Choose Datadog adaptive sampling rates as your service’s sampling strategy.
5. (Optional) Configure explicit sampling rates for specific resources, for which you would like to capture more (for example, 100% of GET /checkout endpoints) or less (for example, 0.1% of /health requests) data.
6. Click Apply.

The configuration should take effect in 5-6 minutes, the time it takes for Datadog to observe the service’s traffic pattern, compute, then apply the sampling rates. Resources that have been configured remotely display as Configured Remote in the Configuration column.

Further reading

추가 유용한 문서, 링크 및 기사:

Ingestion MechanismsDOCUMENTATION Ingestion ControlsDOCUMENTATION