Go Custom Instrumentation using the Datadog API
If you have not yet read the instructions for auto-instrumentation and setup, start with the
Go Setup Instructions.
This page details common use cases for adding and customizing observability with Datadog APM.
Note: This documentation uses v2 of the Go tracer, which Datadog recommends for all users. If you are using v1, see the migration guide to upgrade to v2.
Add custom span tags to your spans to customize your observability within Datadog. The span tags are applied to your incoming traces, allowing you to correlate observed behavior with code-level information such as merchant tier, checkout amount, or user ID.
Add tags directly to a Span interface by calling SetTag:
package main
import (
"log"
"net/http"
"github.com/DataDog/dd-trace-go/v2/ddtrace/tracer"
)
func handler(w http.ResponseWriter, r *http.Request) {
// Create a span for a web request at the /posts URL.
span := tracer.StartSpan("web.request", tracer.ResourceName("/posts"))
defer span.Finish()
// Set tag
span.SetTag("http.url", r.URL.Path)
span.SetTag("<TAG_KEY>", "<TAG_VALUE>")
}
func main() {
tracer.Start(tracer.WithService("<SERVICE_NAME>"))
defer tracer.Stop()
http.HandleFunc("/posts", handler)
log.Fatal(http.ListenAndServe(":8080", nil))
}
Datadog’s integrations make use of the Context type to propagate the current active span.
If you want to add span tags attached to a Context, call the SpanFromContext function:
package main
import (
"net/http"
"github.com/DataDog/dd-trace-go/v2/ddtrace/tracer"
)
func handler(w http.ResponseWriter, r *http.Request) {
// Retrieve a span for a web request attached to a Go Context.
if span, ok := tracer.SpanFromContext(r.Context()); ok {
// Set tag
span.SetTag("http.url", r.URL.Path)
}
}
Add tags to all spans by configuring the tracer with the WithGlobalTag option:
package main
import (
"github.com/DataDog/dd-trace-go/v2/ddtrace/tracer"
)
func main() {
tracer.Start(
tracer.WithGlobalTag("datacenter", "us-1"),
tracer.WithGlobalTag("env", "dev"),
)
defer tracer.Stop()
}
Set errors on a span
To set an error on one of your spans, use tracer.WithError as below:
err := someOperation()
span.Finish(tracer.WithError(err))
Adding spans
If you aren’t using supported library instrumentation (see Library compatibility), you may want to to manually instrument your code.
Unlike other Datadog tracing libraries, when tracing Go applications, it's recommended that you explicitly manage and pass the Go context of your spans. This approach ensures accurate span relationships and meaningful tracing. For more information, see the
Go context library documentation or documentation for any third-party libraries integrated with your application.
Manually creating a new span
To make use of manual instrumentation, use the tracer package which is documented on Datadog’s godoc page (or the v1 godoc page):
There are two functions available to create spans. API details are available for StartSpan here (or here for v1) and for StartSpanFromContext here (or here for v1).
//Create a span with a resource name, which is the child of parentSpan.
span := tracer.StartSpan("mainOp", tracer.ResourceName("/user"), tracer.ChildOf(parentSpan))
// Create a span which will be the child of the span in the Context ctx, if there is a span in the context.
// Returns the new span, and a new context containing the new span.
span, ctx := tracer.StartSpanFromContext(ctx, "mainOp", tracer.ResourceName("/user"))
Asynchronous traces
func main() {
span, ctx := tracer.StartSpanFromContext(context.Background(), "mainOp")
defer span.Finish()
go func() {
asyncSpan := tracer.StartSpanFromContext(ctx, "asyncOp")
defer asyncSpan.Finish()
performOp()
}()
}
Distributed tracing
Create a distributed trace by manually propagating the tracing context:
package main
import (
"net/http"
"github.com/DataDog/dd-trace-go/v2/ddtrace/tracer"
)
func handler(w http.ResponseWriter, r *http.Request) {
span, ctx := tracer.StartSpanFromContext(r.Context(), "post.process")
defer span.Finish()
req, err := http.NewRequest("GET", "http://example.com", nil)
req = req.WithContext(ctx)
// Inject the span Context in the Request headers
err = tracer.Inject(span.Context(), tracer.HTTPHeadersCarrier(req.Header))
if err != nil {
// Handle or log injection error
}
http.DefaultClient.Do(req)
}
Then, on the server side, to continue the trace, start a new Span from the extracted Context:
package main
import (
"net/http"
"github.com/DataDog/dd-trace-go/v2/ddtrace/tracer"
)
func handler(w http.ResponseWriter, r *http.Request) {
// Extract the span Context and continue the trace in this service
sctx, err := tracer.Extract(tracer.HTTPHeadersCarrier(r.Header))
if err != nil {
// Handle or log extraction error
}
span := tracer.StartSpan("post.filter", tracer.ChildOf(sctx))
defer span.Finish()
}
Trace client and Agent configuration
There are additional configurations possible for both the tracing client and Datadog Agent for context propagation with B3 Headers, as well as excluding specific resources from sending traces to Datadog in the event these traces are not wanted in metrics calculated, such as Health Checks.
Propagating context with headers extraction and injection
You can configure the propagation of context for distributed traces by injecting and extracting headers. Read Trace Context Propagation for information.
Resource filtering
Traces can be excluded based on their resource name, to remove synthetic traffic such as health checks from reporting traces to Datadog. This and other security and fine-tuning configurations can be found on the Security page.
Further Reading
Additional helpful documentation, links, and articles:
