If you have not yet read the instructions for auto-instrumentation and setup, start with the
Go Setup Instructions.
This page details common use cases for adding and customizing observability with Datadog APM.
Add custom span tags to your spans to customize your observability within Datadog. The span tags are applied to your incoming traces, allowing you to correlate observed behavior with code-level information such as merchant tier, checkout amount, or user ID.
Add tags directly to a Span
interface by calling SetTag
:
package main
import (
"log"
"net/http"
"gopkg.in/DataDog/dd-trace-go.v1/ddtrace/tracer"
)
func handler(w http.ResponseWriter, r *http.Request) {
// Create a span for a web request at the /posts URL.
span := tracer.StartSpan("web.request", tracer.ResourceName("/posts"))
defer span.Finish()
// Set tag
span.SetTag("http.url", r.URL.Path)
span.SetTag("<TAG_KEY>", "<TAG_VALUE>")
}
func main() {
tracer.Start(tracer.WithService("<SERVICE_NAME>"))
defer tracer.Stop()
http.HandleFunc("/posts", handler)
log.Fatal(http.ListenAndServe(":8080", nil))
}
Datadog’s integrations make use of the Context
type to propagate the current active span.
If you want to add span tags attached to a Context
, call the SpanFromContext
function:
package main
import (
"net/http"
"gopkg.in/DataDog/dd-trace-go.v1/ddtrace/tracer"
)
func handler(w http.ResponseWriter, r *http.Request) {
// Retrieve a span for a web request attached to a Go Context.
if span, ok := tracer.SpanFromContext(r.Context()); ok {
// Set tag
span.SetTag("http.url", r.URL.Path)
}
}
Add tags to all spans by configuring the tracer with the WithGlobalTag
option:
package main
import "gopkg.in/DataDog/dd-trace-go.v1/ddtrace/tracer"
func main() {
tracer.Start(
tracer.WithGlobalTag("datacenter", "us-1"),
tracer.WithGlobalTag("env", "dev"),
)
defer tracer.Stop()
}
Set errors on a span
To set an error on one of your spans, use tracer.WithError
as below:
err := someOperation()
span.Finish(tracer.WithError(err))
Adding spans
If you aren’t using supported library instrumentation (see Library compatibility), you may want to to manually instrument your code.
Manually creating a new span
To make use of manual instrumentation, use the tracer
package which is documented on Datadog’s godoc page:
There are two functions available to create spans. API details are available for StartSpan
here and for StartSpanFromContext
here.
//Create a span with a resource name, which is the child of parentSpan.
span := tracer.StartSpan("mainOp", tracer.ResourceName("/user"), tracer.ChildOf(parentSpan))
// Create a span which will be the child of the span in the Context ctx, if there is a span in the context.
// Returns the new span, and a new context containing the new span.
span, ctx := tracer.StartSpanFromContext(ctx, "mainOp", tracer.ResourceName("/user"))
Asynchronous traces
func main() {
span, ctx := tracer.StartSpanFromContext(context.Background(), "mainOp")
defer span.Finish()
go func() {
asyncSpan := tracer.StartSpanFromContext(ctx, "asyncOp")
defer asyncSpan.Finish()
performOp()
}()
}
Distributed tracing
Create a distributed trace by manually propagating the tracing context:
package main
import (
"net/http"
"gopkg.in/DataDog/dd-trace-go.v1/ddtrace/tracer"
)
func handler(w http.ResponseWriter, r *http.Request) {
span, ctx := tracer.StartSpanFromContext(r.Context(), "post.process")
defer span.Finish()
req, err := http.NewRequest("GET", "http://example.com", nil)
req = req.WithContext(ctx)
// Inject the span Context in the Request headers
err = tracer.Inject(span.Context(), tracer.HTTPHeadersCarrier(req.Header))
if err != nil {
// Handle or log injection error
}
http.DefaultClient.Do(req)
}
Then, on the server side, to continue the trace, start a new Span from the extracted Context
:
package main
import (
"net/http"
"gopkg.in/DataDog/dd-trace-go.v1/ddtrace/tracer"
)
func handler(w http.ResponseWriter, r *http.Request) {
// Extract the span Context and continue the trace in this service
sctx, err := tracer.Extract(tracer.HTTPHeadersCarrier(r.Header))
if err != nil {
// Handle or log extraction error
}
span := tracer.StartSpan("post.filter", tracer.ChildOf(sctx))
defer span.Finish()
}
Trace client and Agent configuration
There are additional configurations possible for both the tracing client and Datadog Agent for context propagation with B3 Headers, as well as excluding specific resources from sending traces to Datadog in the event these traces are not wanted in metrics calculated, such as Health Checks.
B3 headers extraction and injection
The Datadog APM tracer supports B3 headers extraction and injection for distributed tracing.
Distributed headers injection and extraction is controlled by
configuring injection/extraction styles. Two styles are
supported: Datadog
and B3
.
Configure injection styles using the environment variable:
DD_PROPAGATION_STYLE_INJECT=Datadog,B3
Configure extraction styles using the environment variable:
DD_PROPAGATION_STYLE_EXTRACT=Datadog,B3
The values of these environment variables are comma separated lists of
header styles that are enabled for injection or extraction. By default,
the Datadog
extraction style is enabled.
If multiple extraction styles are enabled, extraction attempts are made
in the order that those styles are specified. The first successfully
extracted value is used.
Resource filtering
Traces can be excluded based on their resource name, to remove synthetic traffic such as health checks from reporting traces to Datadog. This and other security and fine-tuning configurations can be found on the Security page.
Further Reading
Additional helpful documentation, links, and articles: