For AI agents: A markdown version of this page is available at https://docs.datadoghq.com/logs/log_configuration/processors/url_parser.md. A documentation index is available at /llms.txt.

URL Parser

Overview

The URL parser processor extracts query parameters and other important parameters from a URL. When set up, the following attributes are produced:

Url Processor

Use cases

The URL Parser is used to extract useful information from a URL, for example, logs coming from nginx can contain URLs that are automatically parsed to extract filters, queries, and sources.

Before and after state of logs

Before:

{
  "client": {
    "ip": "10.12.4.20"
  },
  "http": {
    "method": "GET",
    "url": "https://api.example.com/v1/orders?user_id=12345&limit=20",
    "status_code": 200
  },
  "timestamp": 1696945536000
}

URL Parser

Create a URL Parser processor and configure it to parse the http.url attribute. The processor extracts the URL into multiple structured attributes (scheme, host, path, or query parameters).

After processing:

{
  "client": {
    "ip": "10.12.4.20"
  },
  "http": {
    "method": "GET",
    "url": "https://api.example.com/v1/orders?user_id=12345&limit=20",
    "url_details": {
      "host": "api.example.com",
      "path": "/v1/orders",
      "port": 443,
      "queryString": {
        "user_id": "12345",
        "limit": "20"
      },
      "scheme": "https"
    },
    "status_code": 200
  },
  "timestamp": 1696945536000
}

The URL Parser automatically adds a new nested attribute (http.url_details) containing extracted components of the URL.

API

Use the Datadog Log Pipeline API endpoint with the following URL processor JSON payload:

{
  "type": "url-parser",
  "name": "Parse the URL from http.url attribute.",
  "is_enabled": true,
  "sources": ["http.url"],
  "target": "http.url_details"
}
ParameterTypeRequiredDescription
typeStringYesType of the processor.
nameStringNoName of the processor.
is_enabledBooleanNoIf the processor is enabled or not. Default: false.
sourcesArray of stringsNoArray of source attributes. Default: http.url.
targetStringYesName of the parent attribute that contains all the extracted details from the sources. Default: http.url_details.

Further reading

Additional helpful documentation, links, and articles:

Discover Datadog PipelinesDOCUMENTATION more more