Trace View

View an individual trace to see all of its spans and associated metadata. Each trace can be viewed either as a flame graph or as a list (grouped by service or host).

Trace

Calculate the breakdown of execution time and adjust the color scheme by either service or host.

Service host display

To get a closer look at the flame graph, zoom in by scrolling:

The List view aggregates resources by service and sorts them according to their corresponding count of spans. Services are sorted per relative percentage of execution time spent by the trace in each service:

Trace list

More information

Click on a span in the flame graph to show its metadata below the graph. If there’s an error, the stack trace is provided:

Trace Error

If you are analyzing a trace reporting an error, the error has a specific display if you follow the special meaning tags rules. When submitting your traces you can add attributes to the meta parameter.

Some attributes have special meanings that lead to a dedicated display or specific behavior in Datadog:

AttributeDescription
sql.queryAllows specific SQL query formatting and display in Datadog’s UI.
error.msgAllows dedicated display for error message.
error.typeAllows dedicated display for error types. Available types include, for instance, in Python ValueError or Exception and in Java ClassNotFoundException or NullPointerException.
error.stackAllows a better display of the stack trace of an exception in Datadog’s UI (red boxes, etc…)
Error Formating

View the host information related to the trace including host tags and graphs around the time of the trace.

Trace Host Info

See logs related to your service at the time of the trace. When you hover over a log, a line showing its timestamp is displayed on the trace flame graph. Clicking on the log brings you to the log explorer search.

Trace Logs

Click on a service’s span to see the processes running on its underlying infrastructure. A service’s span processes are correlated with the hosts or pods on which the service runs at the time of the request. You can analyze process metrics such as CPU and RSS memory alongside code-level errors to distinguish between application-specific and wider infrastructure issues. Clicking on a process will bring you to the Live Processes page. To view span-specific processes, enable process collection. Related processes are not currently supported for serverless and browser traces.

Trace Processes

Click on a service’s span to see network dependencies of the service making the request. Use key network performance metrics such as volume, errors (TCP retransmits), and network latency (TCP round-trip time) to differentiate between application-specific and network-wide issues, especially when no code errors have been generated. For instance, you can use network telemetry to determine if high request latency is due to traffic overloading of the relevant application, or faulty dependencies with a downstream pod, security group, or any other tagged endpoint. Clicking on a process brings you to the Network Analytics page. To view span-specific processes, enable Network Performance Monitoring.

Note: Related network telemetry is not currently supported for serverless traces.

Trace Network Dependencies

See attack attempts that target the services of the distributed trace. You can see the pattern used by the attacker, the rule that detects the attack, and whether the attacker found a vulnerability in your service.

Click View in ASM to investigate further using Datadog Application Security Management.

Trace Attack Attempts

Further Reading