---
isPrivate: true
title: (LEGACY) Configurations
description: Datadog, the leading service for cloud-scale monitoring.
breadcrumbs: >-
  Docs > Observability Pipelines > (LEGACY) Observability Pipelines
  Documentation > (LEGACY) Configurations
---

> For the complete documentation index, see [llms.txt](https://docs.datadoghq.com/llms.txt).

# (LEGACY) Configurations

{% callout %}
# Important note for users on the following Datadog sites: app.ddog-gov.com, us2.ddog-gov.com

{% alert level="danger" %}
This product is not supported for your selected [Datadog site](https://docs.datadoghq.com/getting_started/site.md). ({% placeholder "user-datadog-site-name" /%}).
{% /alert %}

{% /callout %}

## Overview{% #overview %}

Observability Pipelines Worker configurations can collect, transform, and route your logs from any source to any destination. The configuration file supports YAML, TOML, and JSON. The three main configuration components are sources, transforms, and sinks.

## Set up an example source{% #set-up-an-example-source %}

[Source components](https://docs.datadoghq.com/observability_pipelines/legacy/reference/sources.md) define how the Observability Pipelines Worker collects or receives data from observability data sources.

Create a YAML configuration file and add the following source example:

{% tab title="YAML" %}

```yaml
sources:
  generate_syslog:
    type: demo_logs
    format: syslog
    count: 100
```

{% /tab %}

{% tab title="TOML" %}

```toml
[sources.generate_syslog]
   type = "demo_logs"
   format = "syslog"
   count = 100
```

{% /tab %}

{% tab title="JSON" %}

```json
"sources": {
    "generate_syslog": {
      "type": "demo_logs",
      "format": "syslog",
      "count": 100
    }
}
```

{% /tab %}

This `source` component has a unique ID of `generate_syslog`. This unique ID is important for transforming and routing the data with the`sink` component.

`type` is the source type from which the Observability Pipelines Worker collects observability data. This example uses a `demo_logs` source, which creates sample log data that enables you to simulate different types of events in various formats. The `format` option tells the `demo_logs` source which type of logs to emit, in this case, Syslog format. The `count` option tells the `demo_logs` source how many lines to emit.

See all supported sources in the [Sources documentation](https://docs.datadoghq.com/observability_pipelines/legacy/reference/sources.md).

## Set up an example transform{% #set-up-an-example-transform %}

Use the following example to define a [transform component](https://docs.datadoghq.com/observability_pipelines/legacy/reference/transforms.md) that manipulates the data collected from the `demo_logs` source.

{% tab title="YAML" %}

```yaml
transforms:
  remap_syslog:
    inputs:
      - generate_syslog
    type: remap
    source: |2
        structured = parse_syslog!(.message)
        . = merge(., structured)
```

{% /tab %}

{% tab title="TOML" %}

```toml
[transforms.remap_syslog]
   inputs = ["generate_syslog" ]
   type = "remap"
   source = '''
     structured = parse_syslog!(.message)
     . = merge(., structured)
'''
```

{% /tab %}

{% tab title="JSON" %}

```json
"transforms": {
    "remap_syslog": {
      "inputs": [
        "generate_syslog"
      ],
      "type": "remap",
      "source": "  structured = parse_syslog!(.message)\n  . = merge(., structured)\n"
    }
  }
```

{% /tab %}

In this `transforms.remap_syslog` component, the `inputs` option is set to `generate_syslog`, which means it receives events from the previously defined `generate_syslog` source. The transform's component type is `remap`.

The `source` contains the list of remapping transformations to apply to each event that the Observability Pipelines Worker receives. In this example, only one operation, `parse_syslog`, is performed, but multiple operations can be added.

The `parse_syslog` function receives a single field called `message`, which contains the Syslog event that is generated in the `generate_syslog` source. This function parses the content of the Syslog-formatted message and emits it as a structured event.

This transform example showcases only a portion of the Observability Pipelines Worker's ability to shape and transform your data\*. See the [Transforms documentation](https://docs.datadoghq.com/observability_pipelines/legacy/reference/transforms.md) for all supported transforms, ranging from sampling, filtering, enrichment, and more.

## Set up an example sink{% #set-up-an-example-sink %}

With the data parsed in the `transform` component, use the following [sink](https://docs.datadoghq.com/observability_pipelines/legacy/reference/sinks.md) example to route the data to a destination.

{% tab title="YAML" %}

```yaml
sinks:
  emit_syslog:
    inputs:
      - remap_syslog
    type: console
    encoding:
      codec: json
```

{% /tab %}

{% tab title="TOML" %}

```toml
[sinks.emit_syslog]
inputs = [ "remap_syslog" ]
type = "console"

  [sinks.emit_syslog.encoding]
  codec = "json"
```

{% /tab %}

{% tab title="JSON" %}

```json
"sinks": {
    "emit_syslog": {
      "inputs": [
        "remap_syslog"
      ],
      "type": "console",
      "encoding": {
        "codec": "json"
      }
    }
}
```

{% /tab %}

This `sink` (or destination) component has the ID of `emit_syslog`. The `inputs` option specifies that the events generated by the `remap_syslog` transform are processed with this sink. The `encoding` option tells the sink to emit the events in JSON format.

See the [Sinks documentation](https://docs.datadoghq.com/observability_pipelines/legacy/reference/sinks.md) for all supported sinks.

## Put it all together{% #put-it-all-together %}

With these three basic components, a source, transform, and sink, you now have a working Observability Pipelines configuration file.

{% tab title="YAML" %}

```yaml
sources:
  generate_syslog:
    type: demo_logs
    format: syslog
    count: 100
transforms:
  remap_syslog:
    inputs:
      - generate_syslog
    type: remap
    source: |2
        structured = parse_syslog!(.message)
        . = merge(., structured)

sinks:
  emit_syslog:
    inputs:
      - remap_syslog
    type: console
    encoding:
      codec: json
```

{% /tab %}

{% tab title="TOML" %}

```toml
[sources.generate_syslog]
type = "demo_logs"
format = "syslog"
count = 100

[transforms.remap_syslog]
inputs = [ "generate_syslog" ]
type = "remap"
source = '''
  structured = parse_syslog!(.message)
  . = merge(., structured)
'''

[sinks.emit_syslog]
inputs = [ "remap_syslog" ]
type = "console"

  [sinks.emit_syslog.encoding]
  codec = "json"
```

{% /tab %}

{% tab title="JSON" %}

```json
{
  "sources": {
    "generate_syslog": {
      "type": "demo_logs",
      "format": "syslog",
      "count": 100
    }
  },
  "transforms": {
    "remap_syslog": {
      "inputs": [
        "generate_syslog"
      ],
      "type": "remap",
      "source": "  structured = parse_syslog!(.message)\n  . = merge(., structured)\n"
    }
  },
  "sinks": {
    "emit_syslog": {
      "inputs": [
        "remap_syslog"
      ],
      "type": "console",
      "encoding": {
        "codec": "json"
      }
    }
  }
}
```

{% /tab %}

Run the following command to compile and run this configuration:

```
vector --config ./<configuration_filename>
```

If successfully setup, the parsed demo logs are printed in JSON format.

## Further Reading{% #further-reading %}

- [Working with data using Observability Pipelines](https://docs.datadoghq.com/observability_pipelines/legacy/working_with_data.md)
- [Set up Observability Pipelines](https://docs.datadoghq.com/observability_pipelines/legacy/setup.md)