For AI agents: A markdown version of this page is available at https://docs.datadoghq.com/logs/log_configuration/processors/array_processor.md. A documentation index is available at /llms.txt.

Overview

Use the array processor to extract, aggregate, or transform values from JSON arrays within your logs.

Supported operations include:

  • Select value from a matching element
  • Compute the length of an array
  • Append a value to an array

Each operation is configured through a dedicated processor.

Define the array processor on the Pipelines page.

Select value from matching element

Extract a specific value from an object inside an array when it matches a condition.

Array processor - Select value from element

Example input:

{
  "httpRequest": {
    "headers": [
      {"name": "Referrer", "value": "https://example.com"},
      {"name": "Accept", "value": "application/json"}
    ]
  }
}

Configuration steps:

  • Array path: httpRequest.headers
  • Condition: name:Referrer
  • Extract value of: value
  • Target attribute: referrer

Result:

{
  "httpRequest": {
    "headers": [...]
  },
  "referrer": "https://example.com"
}

Use the Datadog Log Pipeline API endpoint with the following array processor JSON payload:

{
  "type": "array-processor",
  "name": "Extract Referrer URL",
  "is_enabled": true,
  "operation" : {
    "type" : "select",
    "source": "httpRequest.headers",
    "target": "referrer",
    "filter": "name:Referrer",
    "value_to_extract": "value"
  }
}
ParameterTypeRequiredDescription
typeStringYesType of the processor.
nameStringNoName of the processor.
is_enabledBooleanNoWhether the processor is enabled. Default: false.
operation.typeStringYesType of array processor operation.
operation.sourceStringYesPath of the array you want to select from.
operation.targetStringYesTarget attribute.
operation.filterStringYesExpression to match an array element. The first matching element is selected.
operation.value_to_extractStringYesAttribute to read in the selected element.

Array length

Compute the number of elements in an array.

Array processor - Length

Example input:

{
  "tags": ["prod", "internal", "critical"]
}

Configuration steps:

  • Array attribute: tags
  • Target attribute: tagCount

Result:

{
  "tags": ["prod", "internal", "critical"],
  "tagCount": 3
}

Use the Datadog Log Pipeline API endpoint with the following array processor JSON payload:

{
  "type": "array-processor",
  "name": "Compute number of tags",
  "is_enabled": true,
  "operation" : {
    "type" : "length",
    "source": "tags",
    "target": "tagCount"
  }
}
ParameterTypeRequiredDescription
typeStringYesType of the processor.
nameStringNoName of the processor.
is_enabledBooleanNoWhether the processor is enabled. Default: false.
operation.typeStringYesType of array processor operation.
operation.sourceStringYesPath of the array to extract the length of.
operation.targetStringYesTarget attribute.

Append to array

Add an attribute value to the end of a target array attribute in the log.

Note: If the target array attribute does not exist in the log, it is automatically created.

Array processor - Append

Example input:

{
  "network": {
    "client": {
      "ip": "198.51.100.23"
    }
  },
  "sourceIps": ["203.0.113.1"]
}

Configuration steps:

  • Attribute to append: "network.client.ip"
  • Array attribute to append to: sourceIps

Result:

{
  "network": {
    "client": {
      "ip": "198.51.100.23"
    }
  },
  "sourceIps": ["203.0.113.1", "198.51.100.23"]
}

Use the Datadog Log Pipeline API endpoint with the following array processor JSON payload:

{
  "type": "array-processor",
  "name": "Append client IP to sourceIps",
  "is_enabled": true,
  "operation" : {
    "type" : "append",
    "source": "network.client.ip",
    "target": "sourceIps"
  }
}
ParameterTypeRequiredDescription
typeStringYesType of the processor.
nameStringNoName of the processor.
is_enabledBooleanNoWhether the processor is enabled. Default: false.
operation.typeStringYesType of array processor operation.
operation.sourceStringYesAttribute to append.
operation.targetStringYesArray attribute to append to.
operation.preserve_sourceBooleanNoWhether to preserve the original source after remapping. Default: false.

Further reading

Additional helpful documentation, links, and articles: