Overview

Datadog Database Monitoring (DBM) correlates backend traces from your OpenTelemetry-instrumented application with detailed database performance data. This allows you to link spans from your application to related query metrics and execution plans, helping you identify the exact queries that are slowing down your services.

Requirements

Before you begin, ensure you have configured unified service tagging. This is required for all data correlation in Datadog.

Setup

To correlate traces and metrics, you must:

1. Instrument database spans: Add specific OpenTelemetry attributes to your database spans to enable correlation with DBM.

2. Configure trace ingestion path: Enable the correct feature gate on your Collector or Agent to ensure database spans are properly processed for DBM.

Step 1: Instrument your database spans

For DBM correlation to work, your database spans must include the following attributes.

Using auto instrumentation

To get started, instrument your application using the appropriate OpenTelemetry auto-instrumentation library for your language. For setup instructions, see the official OpenTelemetry instrumentation documentation.

These libraries automatically add the required db.system and db.statement attributes. The Datadog Agent or SDK then derives span.type automatically, so no manual attribute configuration is needed.

processors:
  attributes/add_span_type:
    actions:
      - key: span.type
        value: "sql"
        action: insert
        # Apply this action only to spans that have the db.system attribute
        from_context: span
        when:
          - span.attributes["db.system"] != nil

service:
  pipelines:
    traces:
      # Add the processor to your traces pipeline
      processors: [..., attributes/add_span_type, ...]

Using manual instrumentation

If you are manually creating spans with the OpenTelemetry SDK, you can set the attributes directly in your code. For more information, see the OpenTelemetry documentation.

The following is a conceptual example of manual instrumentation using Python’s OpenTelemetry SDK:

from opentelemetry import trace

tracer = trace.get_tracer("my-app.instrumentation")

# When making a database call, create a span and set attributes
with tracer.start_as_current_span("postgres.query") as span:
    # Set attributes required for DBM correlation
    span.set_attribute("span.type", "sql")
    span.set_attribute("db.system", "postgres")
    span.set_attribute("db.statement", "SELECT * FROM users WHERE id = ?")
    span.set_attribute("db.name", "user_accounts")

    # Your actual database call would go here
    # db_cursor.execute("SELECT * FROM users WHERE id = %s", (user_id,))

Step 2: Configure your ingest path

Depending on how you send traces to Datadog, you may need to enable specific feature gates to ensure database spans are processed correctly.

datadog:
  otelCollector:
    featureGates: "datadog.EnableReceiveResourceSpansV2,datadog.EnableOperationAndResourceNameV2"

otelcontribcol --config=config.yaml \
--feature-gates=datadog.EnableOperationAndResourceNameV2

otelcontribcol --config=config.yaml \
--feature-gates=datadog.EnableReceiveResourceSpansV2,datadog.EnableOperationAndResourceNameV2

View correlated data in Datadog

After your application is sending traces, you can see the correlation in the APM Trace View:

1. Navigate to APM > Traces.
2. Find and click on a trace from your instrumented service.
3. In the trace’s flame graph, select a database span (for example, a span with span.type: sql)
4. In the details panel, click the SQL Queries tab. You should see performance metrics and execution plans for the query.

Correlation levels

DBM correlation for OpenTelemetry traces works at two levels:

• Statement-level correlation: Links a query sample or execution plan to the query signature computed from db.statement. This works automatically after you complete the setup steps above.
• Service-level correlation: Populates the Databases tab and Calling Services list on a database instance page, linking a query to the service that issued it. This requires a SQL comment containing service metadata on the query.

Datadog SDKs add this comment automatically. OpenTelemetry auto-instrumentation doesn’t, so service-level correlation requires adding the comment manually:

/*dddbs='<service>',dde='<env>',ddps='<service>',ddpv='<version>'*/ SELECT * FROM users WHERE id = ?

Without this comment, a database instance shows no calling services. The APM service page shows no database section, even when statement-level correlation is working.

Stored procedures

Correlation isn’t supported for stored procedure calls (for example, CALL my_procedure()). The span records the procedure call itself, while DBM captures the individual SQL statements the procedure runs internally. These two representations don’t match, so DBM can’t link the span to the statements the procedure runs.

Troubleshooting

If you don’t see the expected correlation between your APM traces and DBM, it’s typically due to a missing or incorrect configuration. Check the following common causes:

• Missing attributes: The database span must contain db.system and db.statement. The span.type attribute is also required but is typically derived automatically by Datadog.
• Incorrect unified service tagging: The service tag on your database spans must be set. Verify that unified service tagging is configured correctly.
• The SQL query may not be parsable: The correlation relies on Datadog’s ability to parse the SQL query from the db.statement attribute. If the query uses non-standard or complex syntax, parsing may fail. If you suspect this is the case, contact Datadog support for assistance.
• The correct feature gates must be enabled for your specific trace ingestion path as described in the setup steps.
• No calling services or empty database section on the APM service page: OpenTelemetry doesn’t inject SQL comments automatically. See Correlation levels to add them manually.

Further reading

Additional helpful documentation, links, and articles:

Send OpenTelemetry Traces to DatadogDOCUMENTATION