Remap Reserved Attributes
This product is not supported for your selected
Datadog site. (
).
Overview
Observability Pipelines’ processors enable you to add, edit, and remove log fields. Remapping attributes or rewriting values ensures your logs are processed and standardized properly. For the majority of processing use cases, use the Edit Fields processor to add, remap, or remove fields from your logs. For advanced use cases, use the Custom Processor to conditionally modify fields or rewrite a field’s value.
In Datadog, reserved attributes are log fields that are set aside for specific processing in the platform. Reserved attributes include host
, source
, status
, service
, trace_id
, and message
. Reserved attributes are applied when routing logs to the following Observability Pipelines destinations:
- Datadog Logs
- Amazon S3 (for Log Archives)
- Azure Blob Storage (for Log Archives)
- Google Cloud Storage (for Log Archives)
There are restrictions in Observability Pipelines on how you can modify reserved attributes. For example, reserved attributes cannot be renamed using the Rename Field processor, but must be remapped instead. This guide walks you through the steps to remap the value of reserved attributes.
Remap the value of reserved attributes
To change or override the value of an existing reserved attribute field, Datadog recommends two approaches using Observability Pipelines. The first uses the Edit Fields processor, and the second uses the Custom Processor.
Use an Edit Fields processor for basic field assignments
- Use a Remove field processor to drop the reserved attribute from the log.
- Use an Add field processor to add the reserved attribute back to the log with your correct field name and value assignment.
Note: In terms of the processor order, the Add Field processor should go immediately after the Remove Field processor to ensure correct field remapping.
Example
The Remove field processor image below removes the improperly named service
field from the log.
The Add field processor image below re-adds the service
field back with the correct value.
Use the Custom Processor for dynamic or manual assignments
Use the Custom Processor to rewrite the reserved attribute’s value.
Dynamically assign the value using template syntax to reference another field’s value.
The following Custom Processor script rewrites the service
field and dynamically assigns the value of app_id
to the service
field’s value.
In the below example image, the input shows service
with the value wrongstatus
. After processing the log with the script, the output shows service
with the value of streaming-service
, which is app_id
’s value.
Manually rewrite the value of an attribute with a static name
The following Custom Processor script sets the status
field to the static value info
.
In the below example image, the input shows status
with the value wrongstatus
. After processing the log with the script, the output shows status
with info
as assigned.
Further reading
Additional helpful documentation, links, and articles: