---
title: Reduce Processor
description: >-
  Learn how to use the Reduce processor to group multiple log events into a
  single log based on specified fields and merge strategies.
breadcrumbs: Docs > Observability Pipelines > Processors > Reduce Processor
---

> For the complete documentation index, see [llms.txt](https://docs.datadoghq.com/llms.txt).

# Reduce Processor
Available for:
{% icon name="icon-logs" /%}
 Logs 
## Overview{% #overview %}

The Reduce processor groups multiple log events into a single log, based on the fields specified and the merge strategies selected. Logs are grouped at 10-second intervals. After the interval has elapsed for the group, the reduced log for that group is sent to the next step in the pipeline.

## Setup{% #setup %}

To set up the Reduce processor:

1. Define a filter query. Only logs that match the specified filter query are processed. Reduced logs and logs that do not match the filter query are sent to the next step in the pipeline. See [Search Syntax](https://docs.datadoghq.com/observability_pipelines/search_syntax/logs.md) for more information.
1. In the Group By section, enter the field you want to group the logs by.
1. Click Add Group by Field to add additional fields.
1. In the Merge Strategy section:
   - In On Field, enter the name of the field you want to merge the logs on.
   - Select the merge strategy in the Apply dropdown menu. This is the strategy used to combine events. See the Merge strategies section for descriptions of the available strategies.
   - Click Add Merge Strategy to add additional strategies.

### Merge strategies{% #merge-strategies %}

These are the available merge strategies for combining log events.

| Name           | Description                                                                                                      |
| -------------- | ---------------------------------------------------------------------------------------------------------------- |
| Array          | Appends each value to an array.                                                                                  |
| Concat         | Concatenates each string value, delimited with a space.                                                          |
| Concat newline | Concatenates each string value, delimited with a newline.                                                        |
| Concat raw     | Concatenates each string value, without a delimiter.                                                             |
| Discard        | Discards all values except the first value that was received.                                                    |
| Flat unique    | Creates a flattened array of all unique values that were received.                                               |
| Longest array  | Keeps the longest array that was received.                                                                       |
| Max            | Keeps the maximum numeric value that was received.                                                               |
| Min            | Keeps the minimum numeric value that was received.                                                               |
| Retain         | Discards all values except the last value that was received. Works as a way to coalesce by not retaining `null`. |
| Shortest array | Keeps the shortest array that was received.                                                                      |
| Sum            | Sums all numeric values that were received.                                                                      |

### Set the aggregation window{% #set-the-aggregation-window %}

Optionally, use the `DD_OP_REDUCE_WINDOW_SECS` environment variable to set how long a Reduce processor aggregates events before it sends a reduced log. The variable accepts values from `1` to `300` seconds. The default is `10` seconds.

The setting applies to every Reduce processor on the Worker. Matching events processed by different Workers are not merged because Workers run independently of each other. For example, if event A should be merged with event B but the two events end up on different Workers, the two events are not merged. Therefore, increasing the number of Workers in a pipeline can reduce the number of merges.

A longer aggregation window lets the processor merge events that arrive farther apart, so more events are merged. Each Reduce processor holds its groups of events in memory until the window closes. Memory usage grows with the window size, the number of unique groups, and the number of Reduce processors. If most events are unique, memory can grow substantially and the Worker can run out of memory.

A shorter aggregation window frees memory sooner, which can help high-throughput pipelines. However, fewer events are merged, so more similar events are sent downstream.

Increase the aggregation window gradually and monitor the Worker's memory usage. See [Process metrics](https://docs.datadoghq.com/observability_pipelines/monitoring_and_troubleshooting/pipeline_usage_metrics.md#process-metrics) for the Worker memory metrics.

**Note**: The `DD_OP_REDUCE_WINDOW_SECS` environment variable requires Worker version 2.25.0 or later.

## Health metrics{% #health-metrics %}

For [component metrics](https://docs.datadoghq.com/observability_pipelines/monitoring_and_troubleshooting/pipeline_usage_metrics.md#component-metrics) and [processor buffer metrics](https://docs.datadoghq.com/observability_pipelines/monitoring_and_troubleshooting/pipeline_usage_metrics.md#processor-buffer-metrics) emitted by all processors, see the [Pipelines Usage Metrics](https://docs.datadoghq.com/observability_pipelines/monitoring_and_troubleshooting/pipeline_usage_metrics.md) documentation.

### Reduce metrics{% #reduce-metrics %}

- Use the `component_id` tag to filter or group by individual components.
- The `component_type` tag is `reduce` for these metrics.

{% dl %}

{% dt %}
`pipelines.stale_events_flushed_total`
{% /dt %}

{% dd %}
**Description**: The number of stale events that the processor has flushed.
{% /dd %}

{% dd %}
**Metric type**: count
{% /dd %}

{% /dl %}
