---
title: Correlate Logs and Tests
description: Correlate your logs with your test traces.
breadcrumbs: Docs > Test Optimization in Datadog > Correlate Logs and Tests
---

# Correlate Logs and Tests

{% callout %}
# Important note for users on the following Datadog sites: app.ddog-gov.com, us2.ddog-gov.com

{% alert level="danger" %}
This product is not supported for your selected [Datadog site](https://docs.datadoghq.com/getting_started/site.md). ({% placeholder "user-datadog-site-name" /%}).
{% /alert %}

{% /callout %}

## Overview{% #overview %}

You can correlate Test Optimization data with [logs injected into Datadog](https://docs.datadoghq.com/logs/log_collection.md), which allows you to view and analyze logs for specific test cases.

{% image
   source="https://docs.dd-static.net/images/continuous_integration/correlate_logs_and_tests.4b40b14c588a6ad4d595778ec376cdca.png?auto=format&fit=max&w=850 1x, https://docs.dd-static.net/images/continuous_integration/correlate_logs_and_tests.4b40b14c588a6ad4d595778ec376cdca.png?auto=format&fit=max&w=850&dpr=2 2x"
   alt="Examine logs for specific test cases with logs and tests correlation." /%}

## Setup{% #setup %}

Correlation can be configured differently depending on how you [send your tests data to Datadog](https://docs.datadoghq.com/tests/setup.md).

{% tab title="Cloud CI provider (Agentless)" %}
### Java{% #java %}

Agentless log submission is supported for the following languages and frameworks:

- `dd-trace-java >= 1.35.2` and Log4j2.

Use the following environment variables to enable and configure Agentless log submission:

| Name                                                | Description                                 | Default value |
| --------------------------------------------------- | ------------------------------------------- | ------------- |
| `DD_AGENTLESS_LOG_SUBMISSION_ENABLED` (required)    | Enables/disables log submission             | `false`       |
| `DD_AGENTLESS_LOG_SUBMISSION_LEVEL` (optional)      | Sets log level for Agentless submission     | `INFO`        |
| `DD_AGENTLESS_LOG_SUBMISSION_QUEUE_SIZE` (optional) | Sets the maximum size of pending logs queue | `1024`        |
| `DD_AGENTLESS_LOG_SUBMISSION_URL` (optional)        | Sets custom URL for submitting logs         | -             |

### Javascript/Typescript{% #javascripttypescript %}

Agentless log submission is supported for the following languages and frameworks:

- `dd-trace-js >= 5.24.0` and `dd-trace-js >= 4.48.0` and `winston`.

Use the following environment variables to enable and configure Agentless log submission:

| Name                                             | Description                         | Default value |
| ------------------------------------------------ | ----------------------------------- | ------------- |
| `DD_AGENTLESS_LOG_SUBMISSION_ENABLED` (required) | Enables/disables log submission     | `false`       |
| `DD_AGENTLESS_LOG_SUBMISSION_URL` (optional)     | Sets custom URL for submitting logs | -             |

### .NET{% #net %}

Agentless log submission is supported for the following languages and frameworks:

- `dd-trace-dotnet >= 2.50.0` and XUnit TestOutputHelper.

Use the following environment variables to enable and configure Agentless log submission:

| Name                                      | Description                                   | Default value |
| ----------------------------------------- | --------------------------------------------- | ------------- |
| `DD_CIVISIBILITY_LOGS_ENABLED` (required) | Enables/disables CI Visibility log submission | `false`       |

### Swift{% #swift %}

Use the following environment variables to enable and configure log submission:

| Name                               | Description                            | Default value |
| ---------------------------------- | -------------------------------------- | ------------- |
| `DD_ENABLE_STDOUT_INSTRUMENTATION` | Enables/disables stdout log submission | `false`       |
| `DD_ENABLE_STDERR_INSTRUMENTATION` | Enables/disables stderr log submission | `false`       |

### Python{% #python %}

Requirements: `ddtrace >= 4.8.0`.

Log submission is supported for the pytest test framework, and only when logs are emitted with the standard library `logging` module.

Use the following environment variable to enable log submission for agentless mode:

| Name                                             | Description                     | Default value |
| ------------------------------------------------ | ------------------------------- | ------------- |
| `DD_AGENTLESS_LOG_SUBMISSION_ENABLED` (required) | Enables/disables log submission | `false`       |

If you use the **Datadog Agent** instead of agentless mode, set instead `DD_LOGS_INJECTION=true` in the environment.

#### Out-of-process logs{% #out-of-process-logs %}

When a separate process executes code triggered by a test, it needs a `trace_id` and `span_id` from that test trace to correlate its logs. Use `ddtrace.testing.logs.DDTestLogsHandler` (`ddtrace >= 4.11.0`) to ship those log records to the Datadog logs intake, correlated with the originating test trace.

`DDTestLogsHandler` reads the same environment variables as the pytest plugin to detect the backend (agentless or EVP proxy). It is available in any subprocess where those variables are available.

**Agentless mode** (set `DD_CIVISIBILITY_AGENTLESS_ENABLED=true`):

| Variable     | Description     | Default         |
| ------------ | --------------- | --------------- |
| `DD_API_KEY` | Datadog API key | (required)      |
| `DD_SITE`    | Datadog site    | `datadoghq.com` |

**Agent/EVP proxy mode** (default):

| Variable                  | Description    | Default     |
| ------------------------- | -------------- | ----------- |
| `DD_TRACE_AGENT_URL`      | Full agent URL | -           |
| `DD_TRACE_AGENT_HOSTNAME` | Agent hostname | `localhost` |
| `DD_TRACE_AGENT_PORT`     | Agent port     | `8126`      |

##### Thread-per-worker{% #thread-per-worker %}

For one thread per test worker, use `ThreadLocalCorrelationFilter` to associate each thread's log records with the correct test trace:

```python
import logging
from ddtrace.testing.logs import DDTestLogsHandler, ThreadLocalCorrelationFilter

with DDTestLogsHandler(service="my-service") as handler:
    correlation = ThreadLocalCorrelationFilter()
    handler.addFilter(correlation)
    logging.getLogger().addHandler(handler)

    while True:
        job = queue.get()  # queue and run_test are provided by your worker framework
        correlation.set_context(trace_id=job.trace_id, span_id=job.span_id)
        run_test(job.item)
```

`DDTestLogsHandler` flushes buffered records automatically when used as a context manager. Call `handler.close()` if you do not use the context manager form.

##### Asyncio workers{% #asyncio-workers %}

For asyncio-based workers, `ThreadLocalCorrelationFilter` is not compatible with asyncio-based workers because thread-local storage does not propagate across `asyncio.Task` boundaries. Subclass `CorrelationFilter` and use a `contextvars.ContextVar` instead, which the event loop propagates automatically across `await` boundaries:

```python
import asyncio
import contextvars
import logging
from ddtrace.testing.logs import CorrelationFilter, DDTestLogsHandler

class ContextVarCorrelationFilter(CorrelationFilter):
    def __init__(self):
        super().__init__()
        self._trace_id = contextvars.ContextVar("dd_trace_id", default=None)
        self._span_id = contextvars.ContextVar("dd_span_id", default=None)

    def set_context(self, trace_id, span_id):
        self._trace_id.set(trace_id)
        self._span_id.set(span_id)

    def get_trace_id(self):
        return self._trace_id.get()

    def get_span_id(self):
        return self._span_id.get()

async def run_one(job, correlation):
    correlation.set_context(trace_id=job.trace_id, span_id=job.span_id)
    await run_test(job.item)

async def main(jobs):
    with DDTestLogsHandler(service="my-service") as handler:
        correlation = ContextVarCorrelationFilter()
        handler.addFilter(correlation)
        logging.getLogger().addHandler(handler)
        await asyncio.gather(*(run_one(job, correlation) for job in jobs))
```

### Ruby{% #ruby %}

Agentless logs submission with Test Optimization is supported for Rails applications. Before enabling, ensure that your application is [instrumented with Datadog tracing](https://docs.datadoghq.com/tracing/trace_collection/automatic_instrumentation/dd_libraries/ruby.md#rails-or-hanami-applications).

To use agentless log submission, you need `datadog-ci` version `0.16` or later. The following logging libraries are supported:

- `activesupport >= 5.0` (only when using `ActiveSupport::TaggedLogging`)
- `lograge >= 0.14`
- `semantic_logger >= 4.0`

Use the following environment variable to enable log submission:

| Name                                             | Description                     | Default value |
| ------------------------------------------------ | ------------------------------- | ------------- |
| `DD_AGENTLESS_LOG_SUBMISSION_ENABLED` (required) | Enables/disables log submission | `false`       |

{% /tab %}

{% tab title="On-Premises CI provider (Datadog Agent)" %}

1. [Set up log collection](https://docs.datadoghq.com/logs/log_collection.md) through the Datadog Agent.
1. Follow the steps described in [Correlate Logs and Traces](https://docs.datadoghq.com/tracing/other_telemetry/connect_logs_and_traces.md).

{% /tab %}

## Further Reading{% #further-reading %}

- [Learn about Test Optimization](https://docs.datadoghq.com/tests.md)