---
title: Pipelines Usage Metrics
description: Datadog, the leading service for cloud-scale monitoring.
breadcrumbs: >-
  Docs > Observability Pipelines > Monitoring and Troubleshooting > Pipelines
  Usage Metrics
---

> For the complete documentation index, see [llms.txt](https://docs.datadoghq.com/llms.txt).

# Pipelines Usage Metrics

{% callout %}
# Important note for users on the following Datadog sites: app.ddog-gov.com, us2.ddog-gov.com

{% alert level="danger" %}
This product is not supported for your selected [Datadog site](https://docs.datadoghq.com/getting_started/site.md). ({% placeholder "user-datadog-site-name" /%}).
{% /alert %}

{% /callout %}

## Overview{% #overview %}

This document lists some of the metrics available from Observability Pipelines. You can:

- Create your own [dashboards](https://docs.datadoghq.com/getting_started/dashboards.md), [notebooks](https://docs.datadoghq.com/notebooks.md), and [monitors](https://docs.datadoghq.com/getting_started/monitors.md) with these metrics.
- Use [Metrics Summary](https://app.datadoghq.com/metric/summary) to see metadata and tags available for the metrics. You can also see which dashboards, notebooks, monitors, and SLOs are using those metrics.

See [Getting Started with Tags](https://docs.datadoghq.com/getting_started/tagging.md) for more information on how to use tags to group metrics by specific pipelines, Workers, and components.

All metrics are tagged with the following:

{% dl %}

{% dt %}
`pipeline_id`
{% /dt %}

{% dd %}
The UUID of the pipeline.
{% /dd %}

{% dt %}
`worker_uuid`
{% /dt %}

{% dd %}
The UUID of the Worker emitting the metric.
{% /dd %}

{% dt %}
`op_worker_version`
{% /dt %}

{% dd %}
The version of the Worker emitting the metric.
{% /dd %}

{% dt %}
`rc_version`
{% /dt %}

{% dd %}
The configuration version number, incremented each time the pipeline is updated.
{% /dd %}

{% dt %}
`pipeline_name`
{% /dt %}

{% dd %}
The name of the pipeline when it was last deployed or updated. Available in Worker version 2.18 and later.
{% /dd %}

{% /dl %}

**Notes**:

- Every Worker also runs an internal pipeline that collects the Worker's own telemetry (metrics and logs) and sends it to Datadog. The components in this internal pipeline have a `component_id` tag whose value starts with an underscore (`_`). To exclude these metrics from your queries, use `!component_id:_*`.
- Metrics ending in `_total` report a count for each time interval, so their raw value does not increase monotonically.

## Estimated usage metric{% #estimated-usage-metric %}

{% dl %}

{% dt %}
Observability Pipelines ingested bytes
{% /dt %}

{% dd %}
**Metric**: `datadog.estimated_usage.observability_pipelines.ingested_bytes`
{% /dd %}

{% dd %}
**Description**: The volume of data ingested by Observability Pipelines. See [Estimated Usage Metrics](https://docs.datadoghq.com/account_management/billing/usage_metrics.md) for more information.
{% /dd %}

{% /dl %}

## Host metrics{% #host-metrics %}

These metrics provide information about the host running the Observability Pipelines Worker.

{% dl %}

{% dt %}
Available memory
{% /dt %}

{% dd %}
**Metric**: `pipelines.host.memory_available_bytes`
{% /dd %}

{% dd %}
**Description:** The number of bytes of memory available for new allocations on the host.
{% /dd %}

{% dt %}
Bytes in
{% /dt %}

{% dd %}
**Metric**: `pipelines.host.network_receive_bytes_total`
{% /dd %}

{% dd %}
**Description:** The number of bytes received by the host on all interfaces. Use the `device` tag to filter per interface, for example `device:eth0`.
{% /dd %}

{% dt %}
Bytes out
{% /dt %}

{% dd %}
**Metric**: `pipelines.host.network_transmit_bytes_total`
{% /dd %}

{% dd %}
**Description:** The number of bytes sent by the host on all interfaces. Use the `device` tag to filter per interface.
{% /dd %}

{% dt %}
CPU time
{% /dt %}

{% dd %}
**Metric**: `pipelines.host.cpu_seconds_total`
{% /dd %}

{% dd %}
**Description:** The total CPU time consumed by the host, broken down by mode (user, system, idle, and so on) and CPU core.
{% /dd %}

{% dt %}
Disk read/write bytes
{% /dt %}

{% dd %}
**Metric**: `pipelines.host.disk_read_bytes_total`, `pipelines.host.disk_written_bytes_total`
{% /dd %}

{% dd %}
**Description:** The number of bytes read from and written to all disks on the host.
{% /dd %}

{% dt %}
Host uptime
{% /dt %}

{% dd %}
**Metric**: `pipelines.host.uptime`
{% /dd %}

{% dd %}
**Description:** The amount of time since the host was started, in seconds.
{% /dd %}

{% dt %}
Load average
{% /dt %}

{% dd %}
**Metric**: `pipelines.host.load1`, `pipelines.host.load5`, `pipelines.host.load15`
{% /dd %}

{% dd %}
**Description:** The host's system load average over the last 1, 5, and 15 minutes. Load average is the number of processes that are running or waiting to run, and on Linux also includes processes blocked on uninterruptible I/O. Compare the load average value against the `pipelines.host.logical_cpus` value: a load average value near the CPU count indicates full utilization, and a value above it indicates the host is oversubscribed. Not emitted on Workers running in Windows.
{% /dd %}

{% dt %}
Logical CPUs
{% /dt %}

{% dd %}
**Metric**: `pipelines.host.logical_cpus`
{% /dd %}

{% dd %}
**Description:** The number of logical CPU threads (hardware threads) available on the host.
{% /dd %}

{% dt %}
Total memory
{% /dt %}

{% dd %}
**Metric**: `pipelines.host.memory_total_bytes`
{% /dd %}

{% dd %}
**Description:** The total physical memory (RAM) installed on the host.
{% /dd %}

{% /dl %}

## Process metrics{% #process-metrics %}

These metrics provide information about the Observability Pipelines Worker process.

{% dl %}

{% dt %}
CPU cores allocated
{% /dt %}

{% dd %}
**Metric**: `pipelines.cpu_max_cores`
{% /dd %}

{% dd %}
**Description:** The number of CPU cores allocated to the Worker, as reported by container or cgroup limits.
{% /dd %}

{% dt %}
CPU usage
{% /dt %}

{% dd %}
**Metric**: `pipelines.cpu_usage_seconds_total`
{% /dd %}

{% dd %}
**Description:** The amount of CPU time consumed by the Worker process in seconds (in the user and system space). The rate per second of that metric shows the proportion of the CPU used by the Worker.
{% /dd %}

{% dt %}
Data directory available bytes
{% /dt %}

{% dd %}
**Metric**: `pipelines.data_dir_available_bytes`
{% /dd %}

{% dd %}
**Description:** The free storage space remaining on the filesystem where the Worker stores its buffer and state data. Useful for monitoring disk buffers.
{% /dd %}

{% dt %}
Data directory capacity bytes
{% /dt %}

{% dd %}
**Metric**: `pipelines.data_dir_capacity_bytes`
{% /dd %}

{% dd %}
**Description:** The total storage capacity of the filesystem where the Worker stores its buffer and state data.
{% /dd %}

{% dt %}
Memory limit
{% /dt %}

{% dd %}
**Metric**: `pipelines.memory_max_bytes`
{% /dd %}

{% dd %}
**Description:** The maximum memory the Worker is allowed to use, as set by container or cgroup limits.
{% /dd %}

{% dt %}
Memory usage
{% /dt %}

{% dd %}
**Metric**: `pipelines.resident_memory_used_bytes`
{% /dd %}

{% dd %}
**Description:** The amount of RSS memory used by the Worker process in bytes.
{% /dd %}

{% dt %}
Worker uptime
{% /dt %}

{% dd %}
**Metric**: `pipelines.uptime_seconds`
{% /dd %}

{% dd %}
**Description:** The amount of time since the Worker process was started, in seconds.
{% /dd %}

{% /dl %}

## Worker lifecycle metrics{% #worker-lifecycle-metrics %}

These metrics track Observability Pipelines Worker lifecycle events.

{% dl %}

{% dt %}
Worker reloads
{% /dt %}

{% dd %}
**Metric**: `pipelines.reloaded_total`
{% /dd %}

{% dd %}
**Description:** The number of times the Worker instance has been reloaded, such as after a configuration change.
{% /dd %}

{% dt %}
Worker starts
{% /dt %}

{% dd %}
**Metric**: `pipelines.started_total`
{% /dd %}

{% dd %}
**Description:** The number of times the Worker instance has been started.
{% /dd %}

{% dt %}
Worker stops
{% /dt %}

{% dd %}
**Metric**: `pipelines.stopped_total`
{% /dd %}

{% dd %}
**Description:** The number of times the Worker instance has been stopped.
{% /dd %}

{% /dl %}

## Component metrics{% #component-metrics %}

These metrics are available for sources, processors, and destinations.

- Use the `component_id` tag to filter or group by individual components.
- Use the `component_type` tag to filter or group by the type of source, processor, or destination, such as `quota` for the Quota processor.
- Use the `component_kind` tag to filter or group by `source`, `transform` (processor) or `sink` (destination).

{% tab title="Sources" %}
### Throughput{% #throughput %}

{% dl %}

{% dt %}
Bytes in
{% /dt %}

{% dd %}
**Metric**: `pipelines.component_received_bytes_total`
{% /dd %}

{% dd %}
**Description**: The number of raw bytes read from the source's input, before any decoding or transformation.
{% /dd %}

{% dt %}
Events in
{% /dt %}

{% dd %}
**Metric**: `pipelines.component_received_events_total`
{% /dd %}

{% dd %}
**Description**: The number of events received by the component.
{% /dd %}

{% dt %}
Events out
{% /dt %}

{% dd %}
**Metric**: `pipelines.component_sent_events_total`
{% /dd %}

{% dd %}
**Description**: The number of events the component sends downstream.
{% /dd %}

{% dt %}
Event bytes in
{% /dt %}

{% dd %}
**Metric**: `pipelines.component_received_event_bytes_total`
{% /dd %}

{% dd %}
**Description**: The byte size of events received by the component.
{% /dd %}

{% dt %}
Event bytes out
{% /dt %}

{% dd %}
**Metric**: `pipelines.component_sent_event_bytes_total`
{% /dd %}

{% dd %}
**Description**: The byte size of events the component sends downstream.
{% /dd %}

{% /dl %}

### Errors, data dropped, and timed outs{% #errors-data-dropped-and-timed-outs %}

{% dl %}

{% dt %}
Errors
{% /dt %}

{% dd %}
**Metric**: `pipelines.component_errors_total`
{% /dd %}

{% dd %}
**Description**: The number of errors encountered by the component. Depending on the component, this metric can include an `error_code`, `error_type`, or `reason` tag that describes the error.
{% /dd %}

{% dt %}
Data dropped intentionally or unintentionally
{% /dt %}

{% dd %}
**Metric**: `pipelines.component_discarded_events_total`
{% /dd %}

{% dd %}
**Description**: The number of events dropped. **Note**: To break down this metric, use the `intentional:true` tag to filter for events that are intentionally dropped or the `intentional:false` tag for events that are not intentionally dropped.
{% /dd %}

{% dt %}
Timed out events
{% /dt %}

{% dd %}
**Metric**: `pipelines.component_timed_out_events_total`
{% /dd %}

{% dd %}
**Description**: The number of events that waited more than 5 seconds to be sent to the first processor and resulted in a HTTP 503 error. This could happen when delivery of events are blocked.
{% /dd %}

{% dd %}
**Available for**: HTTP-based sources that have a configured timeout, such as the Datadog Agent.
{% /dd %}

{% dt %}
Timed out requests
{% /dt %}

{% dd %}
**Metric**: `pipelines.component_timed_out_requests_total`
{% /dd %}

{% dd %}
**Description**: The number of requests that timed out for sources that send events to the Worker in batches using HTTP requests.
{% /dd %}

{% dd %}
**Available for**: HTTP-based sources that have a configured timeout, such as the Datadog Agent.
{% /dd %}

{% /dl %}

### Performance{% #performance %}

{% dl %}

{% dt %}
Send latency
{% /dt %}

{% dd %}
**Metric**: `pipelines.source_send_latency_seconds`
{% /dd %}

{% dd %}
**Description**: The time it takes for the source to send a chunk of events to the next component. Available in Worker version 2.16 and later.
{% /dd %}

{% dt %}
Send batch latency
{% /dt %}

{% dd %}
**Metric**: `pipelines.source_send_batch_latency_seconds`
{% /dd %}

{% dd %}
**Description**: The time it takes for the source to send a batch, which can contain multiple event chunks, to the next component. Available in Worker version 2.16 and later.
{% /dd %}

{% dt %}
Source lag time
{% /dt %}

{% dd %}
**Metric**: `pipelines.source_lag_time_seconds`
{% /dd %}

{% dd %}
**Description**: The difference, in seconds, between an event's own timestamp and when the Worker received it. High values indicate stale or delayed data arriving at the pipeline.
{% /dd %}

{% /dl %}

### Buffer{% #buffer %}

Use these metrics to analyze buffer performance. All metrics are emitted on a one-second interval, unless otherwise stated.

These metrics are specific to source buffers, located downstream of a source. Each source emits its own respective buffer metrics. Available in Worker version 2.13 and later. **Note**: Source buffers are not configurable, but these metrics can help monitor backpressure as it propagates to your pipeline's source.

- Use the `component_id` tag to filter or group by individual components.
- Use the `component_type` tag to filter or group by the source type, such as `splunk_hec` for the Splunk HEC source.

{% dl %}

{% dt %}
`pipelines.source_buffer_utilization`
{% /dt %}

{% dd %}
**Description**: Event count in a source's buffer.
{% /dd %}

{% dd %}
**Metric type**: distribution
{% /dd %}

{% dt %}
`pipelines.source_buffer_utilization_level`
{% /dt %}

{% dd %}
**Description**: Event count in a source's buffer.
{% /dd %}

{% dd %}
**Metric type**: gauge
{% /dd %}

{% dt %}
`pipelines.source_buffer_utilization_mean`
{% /dt %}

{% dd %}
**Description**: The exponentially weighted moving average (EWMA) of the number of events in the source's buffer.
{% /dd %}

{% dd %}
**Metric type**: gauge
{% /dd %}

{% dt %}
`pipelines.source_buffer_max_size_events`
{% /dt %}

{% dd %}
**Description**: A source buffer's maximum event capacity.
{% /dd %}

{% dd %}
**Metric type**: gauge
{% /dd %}

{% dt %}
`pipelines.source_buffer_max_size_bytes`
{% /dt %}

{% dd %}
**Description**: A source buffer's maximum byte capacity.
{% /dd %}

{% dd %}
**Metric type**: gauge
{% /dd %}

{% /dl %}

{% /tab %}

{% tab title="Processors" %}
### Throughput{% #throughput %}

{% dl %}

{% dt %}
Events in
{% /dt %}

{% dd %}
**Metric**: `pipelines.component_received_events_total`
{% /dd %}

{% dd %}
**Description**: The number of events received by the component.
{% /dd %}

{% dt %}
Events out
{% /dt %}

{% dd %}
**Metric**: `pipelines.component_sent_events_total`
{% /dd %}

{% dd %}
**Description**: The number of events the component sends downstream.
{% /dd %}

{% dt %}
Event bytes in
{% /dt %}

{% dd %}
**Metric**: `pipelines.component_received_event_bytes_total`
{% /dd %}

{% dd %}
**Description**: The byte size of events received by the component.
{% /dd %}

{% dt %}
Event bytes out
{% /dt %}

{% dd %}
**Metric**: `pipelines.component_sent_event_bytes_total`
{% /dd %}

{% dd %}
**Description**: The byte size of events the component sends downstream.
{% /dd %}

{% dt %}
Events included
{% /dt %}

{% dd %}
**Metric**: `pipelines.included_events_total`
{% /dd %}

{% dd %}
**Description**: The number of events that matched the processor's filter query and were processed. Events that do not match the filter query skip the processor and continue to the next component.
{% /dd %}

{% dt %}
Event bytes included
{% /dt %}

{% dd %}
**Metric**: `pipelines.included_event_bytes_total`
{% /dd %}

{% dd %}
**Description**: The byte size of events that matched the processor's filter query and were processed.
{% /dd %}

{% /dl %}

### Errors and data dropped{% #errors-and-data-dropped %}

{% dl %}

{% dt %}
Errors
{% /dt %}

{% dd %}
**Metric**: `pipelines.component_errors_total`
{% /dd %}

{% dd %}
**Description**: The number of errors encountered by the component. Depending on the component, this metric can include an `error_code`, `error_type`, or `reason` tag that describes the error.
{% /dd %}

{% dt %}
Data dropped intentionally or unintentionally
{% /dt %}

{% dd %}
**Metric**: `pipelines.component_discarded_events_total`
{% /dd %}

{% dd %}
**Description**: The number of events dropped. **Note**: To break down this metric, use the `intentional:true` tag to filter for events that are intentionally dropped or the `intentional:false` tag for events that are not intentionally dropped.
{% /dd %}

{% /dl %}

### Performance{% #performance %}

{% dl %}

{% dt %}
CPU usage
{% /dt %}

{% dd %}
**Metric**: `pipelines.component_cpu_usage_ns_total`
{% /dd %}

{% dd %}
**Description**: The CPU time consumed by a component, in nanoseconds. Use this metric to attribute CPU cost to individual processors. Available in Worker version 2.18 and later for Linux and MacOS.
{% /dd %}

{% dd %}
**Available for these log processors**: - Custom Processor - Dedupe - Enrichment Table - Grok Parser - Parse JSON - Parse XML - Reduce - Remap to OCSF - Sensitive Data Scanner - Split Array - Throttle log processors
{% /dd %}

{% dd %}
**Available for these metrics processors**: - Aggregate - Tag Cardinality Limit metrics
{% /dd %}

{% dt %}
Utilization
{% /dt %}

{% dd %}
**Metric**: `pipelines.utilization`
{% /dd %}

{% dd %}
**Description**: The component's activity. A value of `0` indicates an idle component that is waiting for input. A value close to `1` indicates a component that is never idle, meaning that the component is likely a bottleneck in the processing topology that is creating backpressure. This might cause events to be dropped.
{% /dd %}

{% /dl %}

### Buffer{% #buffer %}

Use these metrics to analyze buffer performance. All metrics are emitted on a one-second interval, unless otherwise stated.

These metrics are specific to processor buffers, located upstream of a processor. Each processor emits its own respective buffer metrics. Available in Worker version 2.13 and later. **Note**: Processor buffers are not configurable, but these metrics can help monitor backpressure as it propagates through your pipeline's processors.

- Use the `component_id` tag to filter or group by individual components.
- Use the `component_type` tag to filter or group by the processor type, such as `quota` for the Quota processor.

{% dl %}

{% dt %}
`pipelines.transform_buffer_utilization`
{% /dt %}

{% dd %}
**Description**: Event count in a processor's buffer.
{% /dd %}

{% dd %}
**Metric type**: distribution
{% /dd %}

{% dt %}
`pipelines.transform_buffer_utilization_level`
{% /dt %}

{% dd %}
**Description**: Event count in a processor's buffer.
{% /dd %}

{% dd %}
**Metric type**: gauge
{% /dd %}

{% dt %}
`pipelines.transform_buffer_utilization_mean`
{% /dt %}

{% dd %}
**Description**: The exponentially weighted moving average (EWMA) of the number of events in a processor's buffer.
{% /dd %}

{% dd %}
**Metric type**: gauge
{% /dd %}

{% dt %}
`pipelines.transform_buffer_max_size_events`
{% /dt %}

{% dd %}
**Description**: A processor buffer's maximum event capacity.
{% /dd %}

{% dd %}
**Metric type**: gauge
{% /dd %}

{% dt %}
`pipelines.transform_buffer_max_size_bytes`
{% /dt %}

{% dd %}
**Description**: A processor buffer's maximum byte capacity.
{% /dd %}

{% dd %}
**Metric type**: gauge
{% /dd %}

{% /dl %}

{% /tab %}

{% tab title="Destinations" %}
### Throughput{% #throughput %}

{% dl %}

{% dt %}
Bytes out
{% /dt %}

{% dd %}
**Metric**: `pipelines.component_sent_bytes_total`
{% /dd %}

{% dd %}
**Description**: The number of raw bytes written to the destination's output, after encoding and transformations.
{% /dd %}

{% dt %}
Events in
{% /dt %}

{% dd %}
**Metric**: `pipelines.component_received_events_total`
{% /dd %}

{% dd %}
**Description**: The number of events received by the component.
{% /dd %}

{% dt %}
Events out
{% /dt %}

{% dd %}
**Metric**: `pipelines.component_sent_events_total`
{% /dd %}

{% dd %}
**Description**: The number of events the component sends downstream.
{% /dd %}

{% dt %}
Event bytes in
{% /dt %}

{% dd %}
**Metric**: `pipelines.component_received_event_bytes_total`
{% /dd %}

{% dd %}
**Description**: The byte size of events received by the component.
{% /dd %}

{% dt %}
Event bytes out
{% /dt %}

{% dd %}
**Metric**: `pipelines.component_sent_event_bytes_total`
{% /dd %}

{% dd %}
**Description**: The byte size of events the component sends downstream.
{% /dd %}

{% /dl %}

### Errors and data dropped{% #errors-and-data-dropped %}

{% dl %}

{% dt %}
Errors
{% /dt %}

{% dd %}
**Metric**: `pipelines.component_errors_total`
{% /dd %}

{% dd %}
**Description**: The number of errors encountered by the component. Depending on the component, this metric can include an `error_code`, `error_type`, or `reason` tag that describes the error.
{% /dd %}

{% dt %}
Data dropped intentionally or unintentionally
{% /dt %}

{% dd %}
**Metric**: `pipelines.component_discarded_events_total`
{% /dd %}

{% dd %}
**Description**: The number of events dropped. **Note**: To break down this metric, use the `intentional:true` tag to filter for events that are intentionally dropped or the `intentional:false` tag for events that are not intentionally dropped.
{% /dd %}

{% /dl %}

### Performance{% #performance %}

{% dl %}

{% dt %}
Utilization
{% /dt %}

{% dd %}
**Metric**: `pipelines.utilization`
{% /dd %}

{% dd %}
**Description**: The component's activity. A value of `0` indicates an idle component that is waiting for input. A value close to `1` indicates a component that is never idle, meaning that the component is likely a bottleneck in the processing topology that is creating backpressure. This might cause events to be dropped.
{% /dd %}

{% /dl %}

### Buffer{% #buffer %}

Use these metrics to analyze buffer performance. All metrics are emitted on a one-second interval, unless otherwise stated.

These metrics are specific to destination buffers, located upstream of a destination. Each destination emits its own respective buffer metrics.

- Use the `component_id` tag to filter or group by individual components.
- Use the `component_type` tag to filter or group by the destination type, such as `datadog_logs` for the Datadog Logs destination.

**Note**: Metrics ending in `_total` report a count for each time interval, so their raw value does not increase monotonically.

{% dl %}

{% dt %}
`pipelines.buffer_size_events`
{% /dt %}

{% dd %}
**Description**: Number of events in a destination's buffer.
{% /dd %}

{% dd %}
**Metric type**: gauge
{% /dd %}

{% dt %}
`pipelines.buffer_size_bytes`
{% /dt %}

{% dd %}
**Description**: Number of bytes in a destination's buffer.
{% /dd %}

{% dd %}
**Metric type**: gauge
{% /dd %}

{% dt %}
`pipelines.buffer_max_size_events`
{% /dt %}

{% dd %}
**Description**: A destination buffer's maximum event capacity.
{% /dd %}

{% dd %}
**Metric type**: gauge
{% /dd %}

{% dt %}
`pipelines.buffer_max_size_bytes`
{% /dt %}

{% dd %}
**Description**: A destination buffer's maximum byte capacity.
{% /dd %}

{% dd %}
**Metric type**: gauge
{% /dd %}

{% dt %}
`pipelines.buffer_send_duration_seconds`
{% /dt %}

{% dd %}
**Description**: The time spent sending a payload to a destination's buffer.
{% /dd %}

{% dd %}
**Metric type**: distribution
{% /dd %}

{% dt %}
`pipelines.buffer_received_events_total`
{% /dt %}

{% dd %}
**Description**: Events received by a destination's buffer.
{% /dd %}

{% dd %}
**Metric type**: counter
{% /dd %}

{% dt %}
`pipelines.buffer_received_bytes_total`
{% /dt %}

{% dd %}
**Description**: Bytes received by a destination's buffer.
{% /dd %}

{% dd %}
**Metric type**: counter
{% /dd %}

{% dt %}
`pipelines.buffer_sent_events_total`
{% /dt %}

{% dd %}
**Description**: Events sent downstream by a destination's buffer.
{% /dd %}

{% dd %}
**Metric type**: counter
{% /dd %}

{% dt %}
`pipelines.buffer_sent_bytes_total`
{% /dt %}

{% dd %}
**Description**: Bytes sent downstream by a destination's buffer.
{% /dd %}

{% dd %}
**Metric type**: counter
{% /dd %}

{% dt %}
`pipelines.buffer_discarded_events_total`
{% /dt %}

{% dd %}
**Description**: Events discarded by the buffer.
{% /dd %}

{% dd %}
**Metric type**: counter
{% /dd %}

{% dd %}
**Additional tags**: `intentional:true` means an incoming event was dropped because the buffer was configured to drop the newest logs when it's full. `intentional:false` means the event was dropped due to an error.
{% /dd %}

{% dt %}
`pipelines.buffer_discarded_bytes_total`
{% /dt %}

{% dd %}
**Description**: Bytes discarded by the buffer.
{% /dd %}

{% dd %}
**Metric type**: counter
{% /dd %}

{% dd %}
**Additional tags**: `intentional:true` means an incoming event was dropped because the buffer was configured to drop the newest logs when it's full. `intentional:false` means the event was dropped due to an error.
{% /dd %}

{% /dl %}

#### Deprecated buffer metrics{% #deprecated-buffer-metrics %}

These metrics are still emitted by the Observability Pipelines Worker for backwards compatibility. Datadog recommends using the replacements when possible.

{% dl %}

{% dt %}
`pipelines.buffer_events`
{% /dt %}

{% dd %}
**Description**: Number of events in a destination's buffer. Use `pipelines.buffer_size_events` instead.
{% /dd %}

{% dd %}
**Metric type**: gauge
{% /dd %}

{% dt %}
`pipelines.buffer_byte_size`
{% /dt %}

{% dd %}
**Description**: Number of bytes in a destination's buffer. Use `pipelines.buffer_size_bytes` instead.
{% /dd %}

{% dd %}
**Metric type**: gauge
{% /dd %}

{% /dl %}

{% /tab %}

## HTTP server metrics{% #http-server-metrics %}

These metrics are emitted by sources that receive data over HTTP, such as the Datadog Agent, HTTP/S Server, OpenTelemetry, and Splunk HEC sources.

- Use the `component_id` tag to filter or group by individual components.
- Use the `component_type` tag to filter or group by the source type.

{% dl %}

{% dt %}
`pipelines.http_server_requests_received_total`
{% /dt %}

{% dd %}
**Description**: The number of HTTP requests received.
{% /dd %}

{% dd %}
**Metric type**: count
{% /dd %}

{% dt %}
`pipelines.http_server_responses_sent_total`
{% /dt %}

{% dd %}
**Description**: The number of HTTP responses sent.
{% /dd %}

{% dd %}
**Metric type**: count
{% /dd %}

{% dt %}
`pipelines.http_server_handler_duration_seconds`
{% /dt %}

{% dd %}
**Description**: The time spent handling an HTTP request.
{% /dd %}

{% dd %}
**Metric type**: distribution
{% /dd %}

{% /dl %}

## HTTP client metrics{% #http-client-metrics %}

These metrics are emitted by destinations that send data over HTTP, including:

- CrowdStrike NG-SIEM
- Datadog Logs
- Datadog Metrics
- Elasticsearch
- Google SecOps
- HTTP Client destination
- Microsoft Sentinel
- New Relic
- OpenSearch
- SentinelOne
- Splunk HEC

**Note**: AWS-based destinations (such as Amazon S3, Amazon OpenSearch, and Amazon Security Lake) do not emit these metrics.

- Use the `component_id` tag to filter or group by individual components.
- Use the `component_type` tag to filter or group by the destination type.

{% dl %}

{% dt %}
`pipelines.http_client_requests_sent_total`
{% /dt %}

{% dd %}
**Description**: The number of HTTP requests sent, tagged by request method.
{% /dd %}

{% dd %}
**Metric type**: count
{% /dd %}

{% dt %}
`pipelines.http_client_responses_total`
{% /dt %}

{% dd %}
**Description**: The number of HTTP responses received, tagged by response status.
{% /dd %}

{% dd %}
**Metric type**: count
{% /dd %}

{% dt %}
`pipelines.http_client_errors_total`
{% /dt %}

{% dd %}
**Description**: The number of HTTP client errors, tagged by error kind.
{% /dd %}

{% dd %}
**Metric type**: count
{% /dd %}

{% dt %}
`pipelines.http_client_rtt_seconds`
{% /dt %}

{% dd %}
**Description**: The round-trip time, in seconds, for HTTP requests, from when the request is sent to when the final response or error is received.
{% /dd %}

{% dd %}
**Metric type**: distribution
{% /dd %}

{% dt %}
`pipelines.http_client_response_rtt_seconds`
{% /dt %}

{% dd %}
**Description**: The round-trip time, in seconds, of HTTP requests, tagged by response status.
{% /dd %}

{% dd %}
**Metric type**: distribution
{% /dd %}

{% dt %}
`pipelines.http_client_error_rtt_seconds`
{% /dt %}

{% dd %}
**Description**: The round-trip time, in seconds, of HTTP requests that resulted in an error, tagged by error kind.
{% /dd %}

{% dd %}
**Metric type**: distribution
{% /dd %}

{% /dl %}

## Adaptive concurrency metrics{% #adaptive-concurrency-metrics %}

These metrics provide information about the adaptive concurrency controller, which automatically tunes how many in-flight HTTP requests a destination allows based on observed response times. They are emitted by destinations that send data over HTTP, including AWS-based destinations.

- Use the `component_id` tag to filter or group by individual components.
- Use the `component_type` tag to filter or group by the destination type.

{% dl %}

{% dt %}
`pipelines.active_endpoints`
{% /dt %}

{% dd %}
**Description**: The number of destination endpoints that are marked healthy.
{% /dd %}

{% dd %}
**Metric type**: gauge
{% /dd %}

{% dt %}
`pipelines.adaptive_concurrency_limit`
{% /dt %}

{% dd %}
**Description**: The concurrency limit for HTTP requests to this destination, automatically adjusted by the adaptive concurrency controller based on response times.
{% /dd %}

{% dd %}
**Metric type**: distribution
{% /dd %}

{% dt %}
`pipelines.adaptive_concurrency_in_flight`
{% /dt %}

{% dd %}
**Description**: The number of HTTP requests in flight to a destination, compared against the adaptive concurrency limit to determine when to throttle.
{% /dd %}

{% dd %}
**Metric type**: distribution
{% /dd %}

{% dt %}
`pipelines.adaptive_concurrency_reached_limit`
{% /dt %}

{% dd %}
**Description**: Whether the adaptive concurrency controller reached its computed limit (`1`) or not (`0`) during the last measurement interval.
{% /dd %}

{% dd %}
**Metric type**: distribution
{% /dd %}

{% dt %}
`pipelines.adaptive_concurrency_back_pressure`
{% /dt %}

{% dd %}
**Description**: Whether the adaptive concurrency controller detected back pressure (`1`) or not (`0`) during the last measurement interval.
{% /dd %}

{% dd %}
**Metric type**: distribution
{% /dd %}

{% dt %}
`pipelines.adaptive_concurrency_averaged_rtt`
{% /dt %}

{% dd %}
**Description**: The smoothed average round-trip time (RTT), in seconds, for HTTP requests to this destination, used as the baseline for adaptive concurrency calculations.
{% /dd %}

{% dd %}
**Metric type**: distribution
{% /dd %}

{% dt %}
`pipelines.adaptive_concurrency_observed_rtt`
{% /dt %}

{% dd %}
**Description**: The round-trip time (RTT), in seconds, observed for the most recent HTTP request to this destination.
{% /dd %}

{% dd %}
**Metric type**: distribution
{% /dd %}

{% dt %}
`pipelines.adaptive_concurrency_past_rtt_mean`
{% /dt %}

{% dd %}
**Description**: The historical mean RTT, in seconds, for HTTP requests to this destination, used as the long-term baseline for adaptive concurrency adjustments.
{% /dd %}

{% dd %}
**Metric type**: distribution
{% /dd %}

{% /dl %}

## Further reading{% #further-reading %}

- [Learn more about the Metrics Summary](https://docs.datadoghq.com/metrics/summary.md)
- [Using the Metrics Explorer to explore and analyze your metrics](https://docs.datadoghq.com/metrics/explorer.md)
- [Getting started with dashboards](https://docs.datadoghq.com/getting_started/dashboards.md)
- [Getting started with monitors](https://docs.datadoghq.com/getting_started/monitors.md)