- 필수 기능
- 시작하기
- Glossary
- 표준 속성
- Guides
- Agent
- 통합
- 개방형텔레메트리
- 개발자
- Administrator's Guide
- API
- Datadog Mobile App
- CoScreen
- Cloudcraft
- 앱 내
- 서비스 관리
- 인프라스트럭처
- 애플리케이션 성능
- APM
- Continuous Profiler
- 스팬 시각화
- 데이터 스트림 모니터링
- 데이터 작업 모니터링
- 디지털 경험
- 소프트웨어 제공
- 보안
- AI Observability
- 로그 관리
- 관리
Trace Context propagation is the mechanism of passing tracing information like Trace ID, Span ID, and sampling decisions from one part of a distributed application to another. This enables all traces (and additional telemetry) in a request to be correlated. When automatic instrumentation is enabled, trace context propagation is handled automatically by the APM SDK.
By default, the Datadog SDK extracts and injects distributed tracing headers using the following formats:
This default configuration maximizes compatibility with older Datadog SDK versions and products while allowing interoperability with other distributed tracing systems like OpenTelemetry.
You may need to customize the trace context propagation configuration if your applications:
Use the following environment variables to configure formats for reading and writing distributed tracing headers. Refer to the Language support section for language-specific configuration values.
DD_TRACE_PROPAGATION_STYLE
datadog,tracecontext
datadog,tracecontext
checks Datadog headers first). The first valid context continues the trace; additional valid contexts become span links.OTEL_PROPAGATORS
Most services send and receive trace context headers using the same format. However, if your service needs to accept trace context headers in one format and send them in another, use these configurations:
DD_TRACE_PROPAGATION_STYLE_EXTRACT
DD_TRACE_PROPAGATION_STYLE_INJECT
The Datadog SDK supports the following trace context formats:
Format | Configuration Value |
---|---|
Datadog | datadog |
W3C Trace Context | tracecontext |
B3 Single | Language Dependent Value |
B3 Multi | b3multi |
Baggage | baggage |
None | none |
The Datadog Java SDK supports the following trace context formats, including deprecated configuration values:
Format | Configuration Value |
---|---|
Datadog | datadog |
W3C Trace Context | tracecontext |
B3 Single | b3 single header |
b3single | |
B3 Multi | b3multi |
b3 (deprecated) | |
AWS X-Ray | xray |
None | none |
In addition to the environment variable configuration, you can also update the propagators using System Property configuration:
-Ddd.trace.propagation.style=datadog,b3multi
-Dotel.propagators=datadog,b3multi
-Ddd.trace.propagation.style.inject=datadog,b3multi
-Ddd.trace.propagation.style.extract=datadog,b3multi
The Datadog Python SDK supports the following trace context formats, including deprecated configuration values:
Format | Configuration Value |
---|---|
Datadog | datadog |
W3C Trace Context | tracecontext |
B3 Single | b3 |
b3 single header (deprecated) | |
B3 Multi | b3multi |
None | none |
The Datadog Ruby SDK supports the following trace context formats, including deprecated configuration values:
Format | Configuration Value |
---|---|
Datadog | datadog |
W3C Trace Context | tracecontext |
B3 Single | b3 |
B3 Multi | b3multi |
None | none |
In addition to the environment variable configuration, you can also update the propagators in code by using Datadog.configure
:
Datadog.configure do |c|
# List of header formats that should be extracted
c.tracing.propagation_extract_style = [ 'tracecontext', 'datadog', 'b3' ]
# List of header formats that should be injected
c.tracing.propagation_inject_style = [ 'tracecontext', 'datadog' ]
end
The Datadog Go SDK supports the following trace context formats, including deprecated configuration values:
Format | Configuration Value |
---|---|
Datadog | datadog |
W3C Trace Context | tracecontext |
B3 Single | B3 single header |
B3 Multi | b3multi |
b3 (deprecated) | |
None | none |
The Datadog Node.js SDK supports the following trace context formats, including deprecated configuration values:
Format | Configuration Value |
---|---|
Datadog | datadog |
W3C Trace Context | tracecontext |
B3 Single | B3 single header |
B3 Multi | b3multi |
B3 (deprecated) | |
None | none |
The Datadog PHP SDK supports the following trace context formats, including deprecated configuration values:
Format | Configuration Value |
---|---|
Datadog | datadog |
W3C Trace Context | tracecontext |
B3 Single | B3 single header |
B3 Multi | b3multi |
B3 (deprecated) | |
None | none |
The following use cases are specific to the Datadog PHP SDK:
When a new PHP script is launched, the Datadog SDK automatically checks for the presence of Datadog headers for distributed tracing:
x-datadog-trace-id
(environment variable: HTTP_X_DATADOG_TRACE_ID
)x-datadog-parent-id
(environment variable: HTTP_X_DATADOG_PARENT_ID
)x-datadog-origin
(environment variable: HTTP_X_DATADOG_ORIGIN
)x-datadog-tags
(environment variable: HTTP_X_DATADOG_TAGS
)To manually set tracing information in a CLI script for new or existing traces, use the DDTrace\set_distributed_tracing_context(string $trace_id, string $parent_id, ?string $origin = null, ?array $tags = null)
function.
<?php
function processIncomingQueueMessage($message) {
}
\DDTrace\trace_function(
'processIncomingQueueMessage',
function(\DDTrace\SpanData $span, $args) {
$message = $args[0];
\DDTrace\set_distributed_tracing_context($message->trace_id, $message->parent_id);
}
);
For version 0.87.0 and later, if the raw headers are available, use the DDTrace\consume_distributed_tracing_headers(array|callable $headersOrCallback)
function. Note: The header names must be in lowercase.
$headers = [
"x-datadog-trace-id" => "1234567890",
"x-datadog-parent-id" => "987654321",
];
\DDTrace\consume_distributed_tracing_headers($headers);
To extract the trace context directly as headers, use the DDTrace\generate_distributed_tracing_headers(?array $inject = null): array
function.
$headers = DDTrace\generate_distributed_tracing_headers();
// Store headers somewhere, inject them in an outbound request, ...
// These $headers can also be read back by \DDTrace\consume_distributed_tracing_headers from another process.
This function’s optional argument accepts an array of injection style names. It defaults to the configured injection style.
The PHP APM SDK supports automatic tracing of the php-amqplib/php-amqplib
library (version 0.87.0+). However, in some cases, your distributed trace may be disconnected. For example, when reading messages from a distributed queue using the basic_get
method outside an existing trace, you need to add a custom trace around the basic_get
call and corresponding message processing:
// Create a surrounding trace
$newTrace = \DDTrace\start_trace_span();
$newTrace->name = 'basic_get.process';
$newTrace->service = 'amqp';
// basic_get call(s) + message(s) processing
$msg = $channel->basic_get($queue);
if ($msg) {
$messageProcessing($msg);
}
// Once done, close the span
\DDTrace\close_span();
Creating this surrounding trace to your consuming-processing logic ensures observability of your distributed queue.
The Datadog C++ SDK supports the following trace context formats, including deprecated configuration values:
Format | Configuration Value |
---|---|
Datadog | datadog |
W3C Trace Context | tracecontext |
B3 Multi | b3 |
b3multi | |
None | none |
In addition to the environment variable configuration, you can also update the propagators in code:
#include <datadog/tracer_config.h>
#include <datadog/propagation_style.h>
namespace dd = datadog::tracing;
int main() {
dd::TracerConfig config;
config.service = "my-service";
// `injection_styles` indicates with which tracing systems trace propagation
// will be compatible when injecting (sending) trace context.
// All styles indicated by `injection_styles` are used for injection.
// `injection_styles` is overridden by the `DD_TRACE_PROPAGATION_STYLE_INJECT`
// and `DD_TRACE_PROPAGATION_STYLE` environment variables.
config.injection_styles = {dd::PropagationStyle::DATADOG, dd::PropagationStyle::B3};
// `extraction_styles` indicates with which tracing systems trace propagation
// will be compatible when extracting (receiving) trace context.
// Extraction styles are applied in the order in which they appear in
// `extraction_styles`. The first style that produces trace context or
// produces an error determines the result of extraction.
// `extraction_styles` is overridden by the
// `DD_TRACE_PROPAGATION_STYLE_EXTRACT` and `DD_TRACE_PROPAGATION_STYLE`
// environment variables.
config.extraction_styles = {dd::PropagationStyle::W3C};
...
}
To extract propagation context, implement a custom DictReader
interface and call Tracer::extract_span
or Tracer::extract_or_create_span
.
Here is an example of extracting propagation context from HTTP Headers:
#include <datadog/dict_reader.h>
#include <datadog/optional.h>
#include <datadog/string_view.h>
#include <unordered_map>
namespace dd = datadog::tracing;
class HTTPHeadersReader : public datadog::tracing::DictReader {
std::unordered_map<dd::StringView, dd::StringView> headers_;
public:
HTTPHeadersReader(std::unordered_map<dd::StringView, dd::StringView> headers)
: headers_(std::move(headers)) {}
~HTTPHeadersReader() override = default;
// Return the value at the specified `key`, or return `nullopt` if there
// is no value at `key`.
dd::Optional<dd::StringView> lookup(dd::StringView key) const override {
auto found = headers_.find(key);
if (found == headers_.cend()) return dd::nullopt;
return found->second;
}
// Invoke the specified `visitor` once for each key/value pair in this object.
void visit(
const std::function<void(dd::StringView key, dd::StringView value)>& visitor)
const override {
for (const auto& [key, value] : headers_) {
visitor(key, value);
}
};
};
// Usage example:
void handle_http_request(const Request& request, datadog::tracing::Tracer& tracer) {
HTTPHeadersReader reader{request.headers};
auto maybe_span = tracer.extract_span(reader);
..
}
To inject propagation context, implement the DictWriter
interface and call Span::inject
on a span instance:
#include <datadog/dict_writer.h>
#include <datadog/string_view.h>
#include <string>
#include <unordered_map>
using namespace dd = datadog::tracing;
class HTTPHeaderWriter : public dd::DictWriter {
std::unordered_map<std::string, std::string>& headers_;
public:
explicit HTTPHeaderWriter(std::unordered_map<std::string, std::string>& headers) : headers_(headers) {}
~HTTPHeaderWriter() override = default;
void set(dd::StringView key, dd::StringView value) override {
headers_.emplace(key, value);
}
};
// Usage example:
void handle_http_request(const Request& request, dd::Tracer& tracer) {
auto span = tracer.create_span();
HTTPHeaderWriter writer(request.headers);
span.inject(writer);
// `request.headers` now populated with the headers needed to propagate the span.
..
}
The Datadog .NET SDK supports the following trace context formats, including deprecated configuration values:
Format | Configuration Value |
---|---|
Datadog | datadog |
W3C Trace Context | tracecontext |
W3C (deprecated) | |
B3 Single | B3 single header |
B3SingleHeader (deprecated) | |
B3 Multi | b3multi |
B3 (deprecated) | |
None | none |
datadog, tracecontext
. This means Datadog headers are used first, followed by W3C Trace Context.tracecontext, Datadog
for both extraction and injection propagation.Datadog
injection style was enabled.DD_TRACE_PROPAGATION_EXTRACT_FIRST=true
configuration specifies whether context extraction should exit immediately upon detecting the first valid tracecontext
. The default value is false
.In most cases, header extraction and injection are automatic. However, there are some known cases where your distributed trace can be disconnected. For instance, when reading messages from a distributed queue, some libraries may lose the span context. It also happens if you set DD_TRACE_KAFKA_CREATE_CONSUMER_SCOPE_ENABLED
to false
when consuming Kafka messages. In these cases, you can add a custom trace using the following code:
var spanContextExtractor = new SpanContextExtractor();
var parentContext = spanContextExtractor.Extract(headers, (headers, key) => GetHeaderValues(headers, key));
var spanCreationSettings = new SpanCreationSettings() { Parent = parentContext };
using var scope = Tracer.Instance.StartActive("operation", spanCreationSettings);
Provide the GetHeaderValues
method. The way this method is implemented depends on the structure that carries SpanContext
.
Here are some examples:
// Confluent.Kafka
IEnumerable<string> GetHeaderValues(Headers headers, string name)
{
if (headers.TryGetLastBytes(name, out var bytes))
{
try
{
return new[] { Encoding.UTF8.GetString(bytes) };
}
catch (Exception)
{
// ignored
}
}
return Enumerable.Empty<string>();
}
// RabbitMQ
IEnumerable<string> GetHeaderValues(IDictionary<string, object> headers, string name)
{
if (headers.TryGetValue(name, out object value) && value is byte[] bytes)
{
return new[] { Encoding.UTF8.GetString(bytes) };
}
return Enumerable.Empty<string>();
}
// SQS
public static IEnumerable<string> GetHeaderValues(IDictionary<string, MessageAttributeValue> headers, string name)
{
// For SQS, there are a maximum of 10 message attribute headers,
// so the Datadog headers are combined into one header with the following properties:
// - Key: "_datadog"
// - Value: MessageAttributeValue object
// - DataType: "String"
// - StringValue: <JSON map with key-value headers>
if (headers.TryGetValue("_datadog", out var messageAttributeValue)
&& messageAttributeValue.StringValue is string jsonString)
{
var datadogDictionary = JsonConvert.DeserializeObject<Dictionary<string, string>>(jsonString);
if (datadogDictionary.TryGetValue(name, out string value))
{
return new[] { value };
}
}
return Enumerable.Empty<string>();
}
When using the SpanContextExtractor
API to trace Kafka consumer spans, set DD_TRACE_KAFKA_CREATE_CONSUMER_SCOPE_ENABLED
to false
. This ensures the consumer span is correctly closed immediately after the message is consumed from the topic, and the metadata (such as partition
and offset
) is recorded correctly. Spans created from Kafka messages using the SpanContextExtractor
API are children of the producer span, and siblings of the consumer span.
If you need to propagate trace context manually (for libraries that are not instrumented automatically, like the WCF client), you can use the SpanContextInjection
API. Here is an example for WCF where this
is the WCF client:
using (OperationContextScope ocs = new OperationContextScope(this.InnerChannel))
{
var spanContextInjector = new SpanContextInjector();
spanContextInjector.Inject(OperationContext.Current.OutgoingMessageHeaders, SetHeaderValues, Tracer.Instance.ActiveScope?.Span?.Context);
}
void SetHeaderValues(MessageHeaders headers, string name, string value)
{
MessageHeader header = MessageHeader.CreateHeader(name, "datadog", value);
headers.Add(header);
}
When the Datadog SDK is configured with the Datadog format for extraction or injection (possibly both), the Datadog SDK interacts with the following request headers:
x-datadog-trace-id
x-datadog-parent-id
x-datadog-origin
rum
, synthetics
, synthetics-browser
.x-datadog-sampling-priority
x-datadog-tags
When the Datadog SDK is configured with the None format for extraction or injection (possibly both), the Datadog SDK does not interact with request headers, meaning that the corresponding context propagation operation does nothing.
Currently available in Python and Node.js. For other languages, please reach out to Support
By default, Baggage is automatically propagated through a distributed request using OpenTelemetry’s W3C-compatible headers. To disable baggage, set DD_TRACE_PROPAGATION_STYLE to datadog,tracecontext
.