This processor parses Extensible Markup Language (XML) so the data can be processed and sent to different destinations. XML is a log format used to store and transport structured data. It is organized in a tree-like structure to represent nested information and uses tags and attributes to define the data. For example, this is XML data using only tags (<recipe>,<type>, and <name>) and no attributes:

<recipe>
    <type>pasta</type>
    <name>Carbonara</name>
</recipe>

This is an XML example where the tag recipe has the attribute type:

<recipe>
    <recipe type="pasta">
    <name>Carbonara</name>
</recipe>

The following image shows a Windows Event 4625 log in XML, next to the same log parsed and output in JSON. By parsing the XML log, the size of the log event was reduced by approximately 30%.

To set up this processor:

1. Define a filter query. Only logs that match the specified filter query are processed. All logs, regardless of whether they match the filter query, are sent to the next step in the pipeline.
2. Enter the path to the log field on which you want to parse XML. Use the path notation <OUTER_FIELD>.<INNER_FIELD> to match subfields. See the Path notation example below.
3. Optionally, in the Enter text key field, input the key name to use for the text node when XML attributes are appended. See the text key example. If the field is left empty, value is used as the key name.
4. Optionally, select Always use text key if you want to store text inside an object using the text key even when no attributes exist.
5. Optionally, toggle Include XML attributes on if you want to include XML attributes. You can then choose to add the attribute prefix you want to use. See attribute prefix example. If the field is left empty, the original attribute key is used.
6. Optionally, select if you want to convert data types into numbers, Booleans, or nulls.
   • If Numbers is selected, numbers are parsed as integers and floats.
   • If Booleans is selected, true and false are parsed as Booleans.
   • If Nulls is selected, the string null is parsed as null.

Path notation example

For the following message structure:

{
    "outer_key": {
        "inner_key": "inner_value",
        "a": {
            "double_inner_key": "double_inner_value",
            "b": "b value"
        },
        "c": "c value"
    },
    "d": "d value"
}

• Use outer_key.inner_key to refer to the key with the value inner_value.
• Use outer_key.inner_key.double_inner_key to refer to the key with the value double_inner_value.

Always use text key example

If Always use text key is selected, the text key is the default (value), and you have the following XML:

<recipe>
    <recipe type="pasta">
    <name>Carbonara</name>
</recipe>

The XML is converted to:

{
    "recipe": {
        "type": "pasta",
        "value": "Carbonara"
        }
}

Text key example

If the key is text and you have the following XML:

<recipe>
    <recipe type="pasta">
    <name>Carbonara</name>
</recipe>

The XML is converted to:

{
    "recipe": {
        "type": "pasta",
        "text": "Carbonara"
        }
}

Attribute prefix example

If you enable Include XML attributes, the attribute is added as a prefix to each XML attribute. For example, if the attribute prefix is @ and you have the following XML:

<recipe type="pasta">Carbonara</recipe>

Then it is converted to the JSON:

{
    "recipe": {
        "@type": "pasta",
        "<text key>": "Carbonara"
        }
}

Sintaxis de las consultas de filtro

Cada procesador tiene una consulta de filtro correspondiente en sus campos. Los procesadores sólo procesan los logs que coinciden con su consulta de filtro. Y en todos los procesadores, excepto el procesador de filtro, los logs que no coinciden con la consulta se envían al siguiente paso de la cadena. Para el procesador de filtro, los logs que no coinciden con la consulta se descartan.

Para cualquier atributo, etiqueta (tag) o par key:value que no sea un atributo reservado, la consulta debe empezar por @. Por el contrario, para filtrar atributos reservados, no es necesario añadir @ delante de la consulta de filtro.

Por ejemplo, para filtrar y descartar logs status:info, tu filtro puede definirse como NOT (status:info). Para filtrar y descartar system-status:info, el filtro debe ser NOT (@system-status:info).

Ejemplos de consulta de filtro:

• NOT (status:debug): Esto filtra sólo los logs que no tienen el estado DEBUG.
• status:ok service:flask-web-app: Esto filtra todos los logs con el estado OK de tu servicioflask-web-app.
  • Esta consulta también se puede escribir como: status:ok AND service:flask-web-app.
• host:COMP-A9JNGYK OR host:COMP-J58KAS: Esta consulta de filtro sólo coincide con los logs de hosts etiquetados.
• @user.status:inactive: Esto filtra los logs con el estado inactive anidado bajo el atributo user.

Las consultas ejecutadas en el worker de Observability Pipelines distinguen entre mayúsculas y minúsculas. Obtén más información sobre cómo escribir consultas de filtro con la sintaxis de búsqueda de logs de Datadog.

Further reading

Más enlaces, artículos y documentación útiles:

Simplify XML log collection and processing with Observability PipelinesBLOG