--- title: Array Processor description: Extract, aggregate, or transform values from JSON arrays within your logs breadcrumbs: Docs > Log Management > Log Configuration > Processors > Array Processor --- > For the complete documentation index, see [llms.txt](https://docs.datadoghq.com/llms.txt). # Array Processor ## Overview{% #overview %} Use the array processor to extract, aggregate, or transform values from JSON arrays within your logs. Supported operations include: - Select value from a matching element - Compute the length of an array - Append a value to an array - Extract key-value pairs from an array Each operation is configured through a dedicated processor. Define the array processor on the [Pipelines page](https://app.datadoghq.com/logs/pipelines). ### Select value from matching element{% #select-value-from-matching-element %} Extract a specific value from an object inside an array when it matches a condition. {% tab title="UI" %} {% image source="https://docs.dd-static.net/images/logs/log_configuration/processor/array_processor_select_value.a3b73e7ca447d0c0dda4f0a124130ce0.png?auto=format&fit=max&w=850 1x, https://docs.dd-static.net/images/logs/log_configuration/processor/array_processor_select_value.a3b73e7ca447d0c0dda4f0a124130ce0.png?auto=format&fit=max&w=850&dpr=2 2x" alt="Array processor - Select value from element" /%} **Example input:** ```json { "httpRequest": { "headers": [ {"name": "Referrer", "value": "https://example.com"}, {"name": "Accept", "value": "application/json"} ] } } ``` **Configuration steps:** - Array path: `httpRequest.headers` - Condition: `name:Referrer` - Extract value of: `value` - Target attribute: `referrer` **Result:** ```json { "httpRequest": { "headers": [...] }, "referrer": "https://example.com" } ``` {% /tab %} {% tab title="API" %} Use the [Datadog Log Pipeline API endpoint](https://docs.datadoghq.com/api/v1/logs-pipelines.md) with the following array processor JSON payload: ```json { "type": "array-processor", "name": "Extract Referrer URL", "is_enabled": true, "operation" : { "type" : "select", "source": "httpRequest.headers", "target": "referrer", "filter": "name:Referrer", "value_to_extract": "value" } } ``` | Parameter | Type | Required | Description | | ---------------------------- | ------- | -------- | ----------------------------------------------------------------------------- | | `type` | String | Yes | Type of the processor. | | `name` | String | No | Name of the processor. | | `is_enabled` | Boolean | No | Whether the processor is enabled. Default: `false`. | | `operation.type` | String | Yes | Type of array processor operation. | | `operation.source` | String | Yes | Path of the array you want to select from. | | `operation.target` | String | Yes | Target attribute. | | `operation.filter` | String | Yes | Expression to match an array element. The first matching element is selected. | | `operation.value_to_extract` | String | Yes | Attribute to read in the selected element. | {% /tab %} ### Array length{% #array-length %} Compute the number of elements in an array. {% tab title="UI" %} {% image source="https://docs.dd-static.net/images/logs/log_configuration/processor/array_processor_length.f5538cc8be08a22cb4aba580720df7c3.png?auto=format&fit=max&w=850 1x, https://docs.dd-static.net/images/logs/log_configuration/processor/array_processor_length.f5538cc8be08a22cb4aba580720df7c3.png?auto=format&fit=max&w=850&dpr=2 2x" alt="Array processor - Length" /%} **Example input:** ```json { "tags": ["prod", "internal", "critical"] } ``` **Configuration steps:** - Array attribute: `tags` - Target attribute: `tagCount` **Result:** ```json { "tags": ["prod", "internal", "critical"], "tagCount": 3 } ``` {% /tab %} {% tab title="API" %} Use the [Datadog Log Pipeline API endpoint](https://docs.datadoghq.com/api/v1/logs-pipelines.md) with the following array processor JSON payload: ```json { "type": "array-processor", "name": "Compute number of tags", "is_enabled": true, "operation" : { "type" : "length", "source": "tags", "target": "tagCount" } } ``` | Parameter | Type | Required | Description | | ------------------ | ------- | -------- | --------------------------------------------------- | | `type` | String | Yes | Type of the processor. | | `name` | String | No | Name of the processor. | | `is_enabled` | Boolean | No | Whether the processor is enabled. Default: `false`. | | `operation.type` | String | Yes | Type of array processor operation. | | `operation.source` | String | Yes | Path of the array to extract the length of. | | `operation.target` | String | Yes | Target attribute. | {% /tab %} ### Append to array{% #append-to-array %} Add an attribute value to the end of a target array attribute in the log. **Note**: If the target array attribute does not exist in the log, it is automatically created. {% tab title="UI" %} {% image source="https://docs.dd-static.net/images/logs/log_configuration/processor/array_processor_append.e11f47cf7e633a5dc975fdb206dd74ce.png?auto=format&fit=max&w=850 1x, https://docs.dd-static.net/images/logs/log_configuration/processor/array_processor_append.e11f47cf7e633a5dc975fdb206dd74ce.png?auto=format&fit=max&w=850&dpr=2 2x" alt="Array processor - Append" /%} **Example input:** ```json { "network": { "client": { "ip": "198.51.100.23" } }, "sourceIps": ["203.0.113.1"] } ``` **Configuration steps:** - Attribute to append: `"network.client.ip"` - Array attribute to append to: `sourceIps` **Result:** ```json { "network": { "client": { "ip": "198.51.100.23" } }, "sourceIps": ["203.0.113.1", "198.51.100.23"] } ``` {% /tab %} {% tab title="API" %} Use the [Datadog Log Pipeline API endpoint](https://docs.datadoghq.com/api/v1/logs-pipelines.md) with the following array processor JSON payload: ```json { "type": "array-processor", "name": "Append client IP to sourceIps", "is_enabled": true, "operation" : { "type" : "append", "source": "network.client.ip", "target": "sourceIps" } } ``` | Parameter | Type | Required | Description | | --------------------------- | ------- | -------- | -------------------------------------------------------------------------- | | `type` | String | Yes | Type of the processor. | | `name` | String | No | Name of the processor. | | `is_enabled` | Boolean | No | Whether the processor is enabled. Default: `false`. | | `operation.type` | String | Yes | Type of array processor operation. | | `operation.source` | String | Yes | Attribute to append. | | `operation.target` | String | Yes | Array attribute to append to. | | `operation.preserve_source` | Boolean | No | Whether to preserve the original source after remapping. Default: `false`. | {% /tab %} ### Extract key-value pairs{% #extract-key-value-pairs %} Flatten an array of key-value objects into individual attributes, creating one attribute per element. Use this operation when a log carries data as a list of `{key, value}` objects, such as the request headers in AWS WAF and HTTP logs. Each entry becomes its own searchable, facetable attribute. Keys are handled automatically, even when they differ from one log to the next. {% tab title="UI" %} {% image source="https://docs.dd-static.net/images/logs/log_configuration/processor/array_processor_key_value.2c631e4da633440c36b84ce47632ee38.png?auto=format&fit=max&w=850 1x, https://docs.dd-static.net/images/logs/log_configuration/processor/array_processor_key_value.2c631e4da633440c36b84ce47632ee38.png?auto=format&fit=max&w=850&dpr=2 2x" alt="Array processor - Extract key-value pairs" /%} **Example input:** ```json { "httpRequest": { "headers": [ {"name": "host", "value": "api.example.com"}, {"name": "user-agent", "value": "curl/8.4.0"}, {"name": "x-forwarded-for", "value": "203.0.113.7"} ] } } ``` **Configuration steps:** - Array path: `httpRequest.headers` - Key attribute: `name` - Value attribute: `value` - Target attribute: (optional) leave blank to add the extracted attributes at the root level of the log **Result:** ```json { "httpRequest": { "headers": [...] }, "host": "api.example.com", "user-agent": "curl/8.4.0", "x-forwarded-for": "203.0.113.7" } ``` {% /tab %} {% tab title="API" %} Use the [Datadog Log Pipeline API endpoint](https://docs.datadoghq.com/api/v1/logs-pipelines.md) with the following array processor JSON payload: ```json { "type": "array-processor", "name": "Extract HTTP headers", "is_enabled": true, "operation" : { "type" : "key-value", "source": "httpRequest.headers", "key_to_extract": "name", "value_to_extract": "value" } } ``` | Parameter | Type | Required | Description | | -------------------------------- | ------- | -------- | ----------------------------------------------------------------------------------------------------------------------------------------- | | `type` | String | Yes | Type of the processor. | | `name` | String | No | Name of the processor. | | `is_enabled` | Boolean | No | Whether the processor is enabled. Default: `false`. | | `operation.type` | String | Yes | Type of array processor operation. | | `operation.source` | String | Yes | Attribute path of the array to extract key-value pairs from. | | `operation.target` | String | No | Attribute that receives the extracted key-value pairs. If not specified, the extracted attributes are added at the root level of the log. | | `operation.key_to_extract` | String | Yes | Key of the attribute in each array element that holds the name to use for the extracted attribute. | | `operation.value_to_extract` | String | Yes | Key of the attribute in each array element that holds the value to use for the extracted attribute. | | `operation.override_on_conflict` | Boolean | No | Whether to override the target attribute if it is already set. Default: `false`. | {% /tab %} ## Further reading{% #further-reading %} Additional helpful documentation, links, and articles: - [Discover Datadog Pipelines](https://docs.datadoghq.com/logs/log_configuration/pipelines.md)