---
title: HTTP Client Source
description: Datadog, the leading service for cloud-scale monitoring.
breadcrumbs: Docs > Observability Pipelines > Sources > HTTP Client Source
---

# HTTP Client Source

{% callout %}
# Important note for users on the following Datadog sites: app.ddog-gov.com

{% alert level="danger" %}
This product is not supported for your selected [Datadog site](https://docs.datadoghq.com/getting_started/site.md). ().
{% /alert %}

{% /callout %}
Available for:
{% icon name="icon-logs" /%}
 Logs 
## Overview{% #overview %}

Use Observability Pipelines' HTTP/S Client source to pull logs from the upstream HTTP/S server.

## Prerequisites{% #prerequisites %}

To use Observability Pipelines' HTTP/S Client source, you need the following information available:

1. The full path of the HTTP Server endpoint that the Observability Pipelines Worker collects log events from. For example, `https://127.0.0.8/logs`.
1. The HTTP authentication token or password.

The HTTP/S Client source pulls data from your upstream HTTP server. Your HTTP server must support GET requests for the HTTP Client endpoint URL that you set as an environment variable when you install the Worker.

## Setup{% #setup %}

Set up this source when you [set up a pipeline](https://docs.datadoghq.com/observability_pipelines/configuration/set_up_pipelines.md). You can set up a pipeline in the [UI](https://app.datadoghq.com/observability-pipelines), using the [API](https://docs.datadoghq.com/api/latest/observability-pipelines.md), or with [Terraform](https://registry.terraform.io/providers/datadog/datadog/latest/docs/resources/observability_pipeline). The instructions in this section are for setting up the source in the UI.

To configure your HTTP/S Client source:

{% alert level="danger" %}
Only enter the identifiers for the HTTP Client endpoint URL and, if applicable, your authorization strategy secrets and TLS key pass. Do not enter the actual values.
{% /alert %}

1. Enter the identifier for your HTTP Client endpoint URL. If you leave it blank, the default is used.
1. Select your authorization strategy. If you selected:
   - **Basic**:
     - Enter the identifier for your HTTP Client username. If you leave it blank, the default is used.
     - Enter the identifier for your HTTP Client password. If you leave it blank, the default is used.
   - **Bearer**: Enter the identifier for your bearer token. If you leave it blank, the default is used.
1. Select the decoder you want to use on the HTTP messages. Logs pulled from the HTTP source must be in this format.

### Optional settings{% #optional-settings %}

#### Enable TLS{% #enable-tls %}

Toggle the switch to **Enable TLS**.

- If you are using Secrets Management, enter the identifier for the key pass. See Set secrets for the default used if the field is left blank.
- The following certificate and key files are required:
  - `Server Certificate Path`: The path to the certificate file that has been signed by your Certificate Authority (CA) root file in DER, PEM, or CRT (X.509).
  - `CA Certificate Path`: The path to the certificate file that is your Certificate Authority (CA) root file in DER, PEM, or CRT (X.509).
  - `Private Key Path`: The path to the `.key` private key file that belongs to your Server Certificate Path in DER, PEM, or CRT (PKCS #8) format.
  - **Notes**:
    - The configuration data directory `/var/lib/observability-pipelines-worker/config/` is automatically appended to the file paths. See [Advanced Worker Configurations](https://docs.datadoghq.com/observability_pipelines/configuration/install_the_worker/advanced_worker_configurations.md) for more information.
    - The file must be readable by the `observability-pipelines-worker` group and user.

#### Scrape settings{% #scrape-settings %}

- Enter the interval between scrapes.
  - Your HTTP Server must be able to handle GET requests at this interval.
  - Since requests run concurrently, if a scrape takes longer than the interval given, a new scrape is started, which can consume extra resources. Set the timeout to a value lower than the scrape interval to prevent this from happening.
- Enter the timeout for each scrape request.

## Set secrets{% #set-secrets %}

These are the defaults used for secret identifiers and environment variables.

**Note**: If you enter secret identifiers and then choose to use environment variables, the environment variable is the identifier entered and prepended with `DD_OP`. For example, if you entered `PASSWORD_1` for a password identifier, the environment variable for that password is `DD_OP_PASSWORD_1`.

{% tab title="Secrets Management" %}

- HTTP Client endpoint URL identifier:
  - References the endpoint from which the Observability Pipelines Worker collects log events.
  - The default identifier is `SOURCE_HTTP_CLIENT_ENDPOINT_URL`.
- HTTP Client TLS passphrase identifier (when TLS is enabled):
  - The default identifier is `SOURCE_HTTP_CLIENT_KEY_PASS`.
- If you are using basic authentication:
  - HTTP Client username identifier:
    - The default identifier is `SOURCE_HTTP_CLIENT_USERNAME`.
  - HTTP Client password identifier:
    - The default identifier is `SOURCE_HTTP_CLIENT_PASSWORD`.
- If you are using bearer authentication:
  - HTTP Client bearer token identifier:
    - The default identifier is `SOURCE_HTTP_CLIENT_BEARER_TOKEN`.

{% /tab %}

{% tab title="Environment Variables" %}

- HTTP/s endpoint URL:
  - The Observability Pipelines Worker collects log events from this endpoint. For example, `https://127.0.0.8/logs`.
  - The default environment variable is `DD_OP_SOURCE_HTTP_CLIENT_ENDPOINT_URL`.
- HTTP/S Client TLS passphrase (when enabled):
  - The default environment variable is `DD_OP_SOURCE_HTTP_CLIENT_KEY_PASS`.
- If you are using basic authentication:
  - HTTP/S endpoint authentication username and password.
  - The default environment variable is `DD_OP_SOURCE_HTTP_CLIENT_USERNAME` and `DD_OP_SOURCE_HTTP_CLIENT_PASSWORD`.
- If you are using bearer authentication:
  - HTTP/S endpoint bearer token.
  - The default environment variable is `DD_OP_SOURCE_HTTP_CLIENT_BEARER_TOKEN`.

{% /tab %}