--- isPrivate: true title: Sources description: Datadog, the leading service for cloud-scale monitoring. breadcrumbs: >- Docs > Observability Pipelines > (LEGACY) Observability Pipelines Documentation > Reference > Sources --- > For the complete documentation index, see [llms.txt](https://docs.datadoghq.com/llms.txt). # (LEGACY) Sources A source is where data is collected and sent to Observability Pipelines. The source component in a configuration defines how Observability Pipelines collects or receives data from the source. ### AMQP{% #amqpsource %} Supports AMQP version 0.9.1 OptionsSchema {% tab title="amqpsource-request-model" %} FieldrequiredTypeDescription acknowledgements optional object Controls how acknowledgements are handled by this source. **DEPRECATED**: This setting is **deprecated** in favor of enabling `acknowledgements` at the [global](https://vector.dev/docs/reference/configuration/global-options/#acknowledgements) or sink level. Enabling or disabling acknowledgements at the source level has **no effect** on acknowledgement behavior. See [End-to-end Acknowledgements](https://vector.dev/docs/about/under-the-hood/architecture/end-to-end-acknowledgements/) for more information on how event acknowledgement is handled. enabled optional boolean,​null Whether or not end-to-end acknowledgements are enabled for this source. default: nullconsumer optional string The identifier for the consumer. decoding optional Configures how events are decoded from raw bytes. Bytes optional object Uses the raw bytes as-is. codec required bytes Uses the raw bytes as-is. Json optional Decodes the raw bytes as [JSON](https://www.json.org/). json optional object JSON-specific decoding options. lossy optional boolean Determines whether or not to replace invalid UTF-8 sequences instead of failing. When true, invalid UTF-8 sequences are replaced with the [`U+FFFD REPLACEMENT CHARACTER`](https://en.wikipedia.org/wiki/Specials_\(Unicode_block\)#Replacement_character). default: truecodec required json Decodes the raw bytes as [JSON](https://www.json.org/). Protobuf optional Decodes the raw bytes as [protobuf](https://protobuf.dev/). protobuf optional object Protobuf-specific decoding options. desc_file required string Path to desc file message_type required string message type. e.g package.message codec required protobuf Decodes the raw bytes as [protobuf](https://protobuf.dev/). Syslog optional Decodes the raw bytes as a Syslog message. Decodes either as the [RFC 3164](https://www.ietf.org/rfc/rfc3164.txt)-style format ("old" style) or the [RFC 5424](https://www.ietf.org/rfc/rfc5424.txt)-style format ("new" style, includes structured data). syslog optional object Syslog-specific decoding options. lossy optional boolean Determines whether or not to replace invalid UTF-8 sequences instead of failing. When true, invalid UTF-8 sequences are replaced with the [`U+FFFD REPLACEMENT CHARACTER`](https://en.wikipedia.org/wiki/Specials_\(Unicode_block\)#Replacement_character). default: truecodec required syslog Decodes the raw bytes as a Syslog message. Decodes either as the [RFC 3164](https://www.ietf.org/rfc/rfc3164.txt)-style format ("old" style) or the [RFC 5424](https://www.ietf.org/rfc/rfc5424.txt)-style format ("new" style, includes structured data). Native optional object Decodes the raw bytes as Vector's [native Protocol Buffers format](https://github.com/vectordotdev/vector/blob/master/lib/vector-core/proto/event.proto). This codec is **[experimental](https://vector.dev/highlights/2022-03-31-native-event-codecs)**. codec required native Decodes the raw bytes as Vector's [native Protocol Buffers format](https://github.com/vectordotdev/vector/blob/master/lib/vector-core/proto/event.proto). This codec is **[experimental](https://vector.dev/highlights/2022-03-31-native-event-codecs)**. NativeJson optional Decodes the raw bytes as Vector's [native JSON format](https://github.com/vectordotdev/vector/blob/master/lib/codecs/tests/data/native_encoding/schema.cue). This codec is **[experimental](https://vector.dev/highlights/2022-03-31-native-event-codecs)**. native_json optional object Vector's native JSON-specific decoding options. lossy optional boolean Determines whether or not to replace invalid UTF-8 sequences instead of failing. When true, invalid UTF-8 sequences are replaced with the [`U+FFFD REPLACEMENT CHARACTER`](https://en.wikipedia.org/wiki/Specials_\(Unicode_block\)#Replacement_character). default: truecodec required native_json Decodes the raw bytes as Vector's [native JSON format](https://github.com/vectordotdev/vector/blob/master/lib/codecs/tests/data/native_encoding/schema.cue). This codec is **[experimental](https://vector.dev/highlights/2022-03-31-native-event-codecs)**. Gelf optional Decodes the raw bytes as a [GELF](https://docs.graylog.org/docs/gelf) message. gelf optional object GELF-specific decoding options. lossy optional boolean Determines whether or not to replace invalid UTF-8 sequences instead of failing. When true, invalid UTF-8 sequences are replaced with the [`U+FFFD REPLACEMENT CHARACTER`](https://en.wikipedia.org/wiki/Specials_\(Unicode_block\)#Replacement_character). default: truecodec required gelf Decodes the raw bytes as a [GELF](https://docs.graylog.org/docs/gelf) message. exchange_key optional string The `AMQP` exchange key. framing optional Framing configuration. Framing handles how events are separated when encoded in a raw byte form, where each event is a frame that must be prefixed, or delimited, in a way that marks where an event begins and ends within the byte stream. Bytes optional object Byte frames are passed through as-is according to the underlying I/O boundaries (for example, split between messages or stream segments). method required bytes Byte frames are passed through as-is according to the underlying I/O boundaries (for example, split between messages or stream segments). CharacterDelimited optional Byte frames which are delimited by a chosen character. character_delimited required object Options for the character delimited decoder. delimiter required integer The character that delimits byte sequences. max_length optional integer,​null The maximum length of the byte buffer. This length does *not* include the trailing delimiter. By default, there is no maximum length enforced. If events are malformed, this can lead to additional resource usage as events continue to be buffered in memory, and can potentially lead to memory exhaustion in extreme cases. If there is a risk of processing malformed data, such as logs with user-controlled input, consider setting the maximum length to a reasonably large value as a safety net. This ensures that processing is not actually unbounded. method required character_delimited Byte frames which are delimited by a chosen character. LengthDelimited optional object Byte frames which are prefixed by an unsigned big-endian 32-bit integer indicating the length. method required length_delimited Byte frames which are prefixed by an unsigned big-endian 32-bit integer indicating the length. NewlineDelimited optional Byte frames which are delimited by a newline character. newline_delimited optional object Options for the newline delimited decoder. max_length optional integer,​null The maximum length of the byte buffer. This length does *not* include the trailing delimiter. By default, there is no maximum length enforced. If events are malformed, this can lead to additional resource usage as events continue to be buffered in memory, and can potentially lead to memory exhaustion in extreme cases. If there is a risk of processing malformed data, such as logs with user-controlled input, consider setting the maximum length to a reasonably large value as a safety net. This ensures that processing is not actually unbounded. method required newline_delimited Byte frames which are delimited by a newline character. OctetCounting optional Byte frames according to the [octet counting](https://tools.ietf.org/html/rfc6587#section-3.4.1) format. octet_counting optional object Options for the octet counting decoder. max_length optional integer,​null The maximum length of the byte buffer. method required octet_counting Byte frames according to the [octet counting](https://tools.ietf.org/html/rfc6587#section-3.4.1) format. log_namespace optional boolean,​null The namespace to use for logs. This overrides the global setting. offset_key optional string The `AMQP` offset key. queue optional string The name of the queue to consume. routing_key_field optional string The `AMQP` routing key. connection_string required string URI for the AMQP server. The URI has the format of `amqp://:@:/?timeout=`. The default vhost can be specified by using a value of `%2f`. To connect over TLS, a scheme of `amqps` can be specified instead. For example, `amqps://...`. Additional TLS settings, such as client certificate verification, can be configured under the `tls` section. tls optional TLS configuration. Option 1 optional object TLS configuration. alpn_protocols optional array,​null Sets the list of supported ALPN protocols. Declare the supported ALPN protocols, which are used during negotiation with peer. They are prioritized in the order that they are defined. ca_file optional Absolute path to an additional CA certificate file. The certificate must be in the DER or PEM (X.509) format. Additionally, the certificate can be provided as an inline string in PEM format. Option 1 optional string A file path. crt_file optional Absolute path to a certificate file used to identify this server. The certificate must be in DER, PEM (X.509), or PKCS#12 format. Additionally, the certificate can be provided as an inline string in PEM format. If this is set, and is not a PKCS#12 archive, `key_file` must also be set. Option 1 optional string A file path. key_file optional Absolute path to a private key file used to identify this server. The key must be in DER or PEM (PKCS#8) format. Additionally, the key can be provided as an inline string in PEM format. Option 1 optional string A file path. key_pass optional string,​null Passphrase used to unlock the encrypted key file. This has no effect unless `key_file` is set. verify_certificate optional boolean,​null Enables certificate verification. If enabled, certificates must not be expired and must be issued by a trusted issuer. This verification operates in a hierarchical manner, checking that the leaf certificate (the certificate presented by the client/server) is not only valid, but that the issuer of that certificate is also valid, and so on until the verification process reaches a root certificate. Relevant for both incoming and outgoing connections. Do NOT set this to `false` unless you understand the risks of not verifying the validity of certificates. verify_hostname optional boolean,​null Enables hostname verification. If enabled, the hostname used to connect to the remote host must be present in the TLS certificate presented by the remote host, either as the Common Name or as an entry in the Subject Alternative Name extension. Only relevant for outgoing connections. Do NOT set this to `false` unless you understand the risks of not verifying the remote hostname. {% /tab %} {% tab title="amqpsource-request-example" %} ```yaml acknowledgements: enabled: null consumer: vector decoding: codec: bytes exchange_key: exchange framing: method: bytes log_namespace: boolean offset_key: offset queue: vector routing_key_field: routing type: amqp ``` {% /tab %} ### AWS Kinesis Firehose{% #awskinesisfirehose %} Configuration for the `aws_kinesis_firehose` source. OptionsSchema {% tab title="awskinesisfirehose-request-model" %} FieldrequiredTypeDescription access_key optional An access key to authenticate requests against. **DEPRECATED**: AWS Kinesis Firehose can be configured to pass along a user-configurable access key with each request. If configured, `access_key` should be set to the same value. Otherwise, all requests are allowed. Option 1 optional string Wrapper for sensitive strings containing credentials access_keys optional array,​null A list of access keys to authenticate requests against. AWS Kinesis Firehose can be configured to pass along a user-configurable access key with each request. If configured, `access_keys` should be set to the same value. Otherwise, all requests are allowed. acknowledgements optional object Controls how acknowledgements are handled by this source. **DEPRECATED**: This setting is **deprecated** in favor of enabling `acknowledgements` at the [global](https://vector.dev/docs/reference/configuration/global-options/#acknowledgements) or sink level. Enabling or disabling acknowledgements at the source level has **no effect** on acknowledgement behavior. See [End-to-end Acknowledgements](https://vector.dev/docs/about/under-the-hood/architecture/end-to-end-acknowledgements/) for more information on how event acknowledgement is handled. enabled optional boolean,​null Whether or not end-to-end acknowledgements are enabled for this source. default: nulladdress required string The socket address to listen for connections on. decoding optional Configures how events are decoded from raw bytes. Bytes optional object Uses the raw bytes as-is. codec required bytes Uses the raw bytes as-is. Json optional Decodes the raw bytes as [JSON](https://www.json.org/). json optional object JSON-specific decoding options. lossy optional boolean Determines whether or not to replace invalid UTF-8 sequences instead of failing. When true, invalid UTF-8 sequences are replaced with the [`U+FFFD REPLACEMENT CHARACTER`](https://en.wikipedia.org/wiki/Specials_\(Unicode_block\)#Replacement_character). default: truecodec required json Decodes the raw bytes as [JSON](https://www.json.org/). Protobuf optional Decodes the raw bytes as [protobuf](https://protobuf.dev/). protobuf optional object Protobuf-specific decoding options. desc_file required string Path to desc file message_type required string message type. e.g package.message codec required protobuf Decodes the raw bytes as [protobuf](https://protobuf.dev/). Syslog optional Decodes the raw bytes as a Syslog message. Decodes either as the [RFC 3164](https://www.ietf.org/rfc/rfc3164.txt)-style format ("old" style) or the [RFC 5424](https://www.ietf.org/rfc/rfc5424.txt)-style format ("new" style, includes structured data). syslog optional object Syslog-specific decoding options. lossy optional boolean Determines whether or not to replace invalid UTF-8 sequences instead of failing. When true, invalid UTF-8 sequences are replaced with the [`U+FFFD REPLACEMENT CHARACTER`](https://en.wikipedia.org/wiki/Specials_\(Unicode_block\)#Replacement_character). default: truecodec required syslog Decodes the raw bytes as a Syslog message. Decodes either as the [RFC 3164](https://www.ietf.org/rfc/rfc3164.txt)-style format ("old" style) or the [RFC 5424](https://www.ietf.org/rfc/rfc5424.txt)-style format ("new" style, includes structured data). Native optional object Decodes the raw bytes as Vector's [native Protocol Buffers format](https://github.com/vectordotdev/vector/blob/master/lib/vector-core/proto/event.proto). This codec is **[experimental](https://vector.dev/highlights/2022-03-31-native-event-codecs)**. codec required native Decodes the raw bytes as Vector's [native Protocol Buffers format](https://github.com/vectordotdev/vector/blob/master/lib/vector-core/proto/event.proto). This codec is **[experimental](https://vector.dev/highlights/2022-03-31-native-event-codecs)**. NativeJson optional Decodes the raw bytes as Vector's [native JSON format](https://github.com/vectordotdev/vector/blob/master/lib/codecs/tests/data/native_encoding/schema.cue). This codec is **[experimental](https://vector.dev/highlights/2022-03-31-native-event-codecs)**. native_json optional object Vector's native JSON-specific decoding options. lossy optional boolean Determines whether or not to replace invalid UTF-8 sequences instead of failing. When true, invalid UTF-8 sequences are replaced with the [`U+FFFD REPLACEMENT CHARACTER`](https://en.wikipedia.org/wiki/Specials_\(Unicode_block\)#Replacement_character). default: truecodec required native_json Decodes the raw bytes as Vector's [native JSON format](https://github.com/vectordotdev/vector/blob/master/lib/codecs/tests/data/native_encoding/schema.cue). This codec is **[experimental](https://vector.dev/highlights/2022-03-31-native-event-codecs)**. Gelf optional Decodes the raw bytes as a [GELF](https://docs.graylog.org/docs/gelf) message. gelf optional object GELF-specific decoding options. lossy optional boolean Determines whether or not to replace invalid UTF-8 sequences instead of failing. When true, invalid UTF-8 sequences are replaced with the [`U+FFFD REPLACEMENT CHARACTER`](https://en.wikipedia.org/wiki/Specials_\(Unicode_block\)#Replacement_character). default: truecodec required gelf Decodes the raw bytes as a [GELF](https://docs.graylog.org/docs/gelf) message. framing optional Framing configuration. Framing handles how events are separated when encoded in a raw byte form, where each event is a frame that must be prefixed, or delimited, in a way that marks where an event begins and ends within the byte stream. Bytes optional object Byte frames are passed through as-is according to the underlying I/O boundaries (for example, split between messages or stream segments). method required bytes Byte frames are passed through as-is according to the underlying I/O boundaries (for example, split between messages or stream segments). CharacterDelimited optional Byte frames which are delimited by a chosen character. character_delimited required object Options for the character delimited decoder. delimiter required integer The character that delimits byte sequences. max_length optional integer,​null The maximum length of the byte buffer. This length does *not* include the trailing delimiter. By default, there is no maximum length enforced. If events are malformed, this can lead to additional resource usage as events continue to be buffered in memory, and can potentially lead to memory exhaustion in extreme cases. If there is a risk of processing malformed data, such as logs with user-controlled input, consider setting the maximum length to a reasonably large value as a safety net. This ensures that processing is not actually unbounded. method required character_delimited Byte frames which are delimited by a chosen character. LengthDelimited optional object Byte frames which are prefixed by an unsigned big-endian 32-bit integer indicating the length. method required length_delimited Byte frames which are prefixed by an unsigned big-endian 32-bit integer indicating the length. NewlineDelimited optional Byte frames which are delimited by a newline character. newline_delimited optional object Options for the newline delimited decoder. max_length optional integer,​null The maximum length of the byte buffer. This length does *not* include the trailing delimiter. By default, there is no maximum length enforced. If events are malformed, this can lead to additional resource usage as events continue to be buffered in memory, and can potentially lead to memory exhaustion in extreme cases. If there is a risk of processing malformed data, such as logs with user-controlled input, consider setting the maximum length to a reasonably large value as a safety net. This ensures that processing is not actually unbounded. method required newline_delimited Byte frames which are delimited by a newline character. OctetCounting optional Byte frames according to the [octet counting](https://tools.ietf.org/html/rfc6587#section-3.4.1) format. octet_counting optional object Options for the octet counting decoder. max_length optional integer,​null The maximum length of the byte buffer. method required octet_counting Byte frames according to the [octet counting](https://tools.ietf.org/html/rfc6587#section-3.4.1) format. log_namespace optional boolean,​null The namespace to use for logs. This overrides the global setting. record_compression optional The compression scheme to use for decompressing records within the Firehose message. Some services, like AWS CloudWatch Logs, [compresses the events with gzip](https://docs.aws.amazon.com/firehose/latest/dev/writing-with-cloudwatch-logs.html), before sending them AWS Kinesis Firehose. This option can be used to automatically decompress them before forwarding them to the next component. Note that this is different from [Content encoding option](https://docs.aws.amazon.com/firehose/latest/dev/create-destination.html#create-destination-http) of the Firehose HTTP endpoint destination. That option controls the content encoding of the entire HTTP request. auto optional auto Automatically attempt to determine the compression scheme. The compression scheme of the object is determined by looking at its file signature, also known as [magic bytes](https://en.wikipedia.org/wiki/List_of_file_signatures). If the record fails to decompress with the discovered format, the record is forwarded as is. Thus, if you know the records are always gzip encoded (for example, if they are coming from AWS CloudWatch Logs), set `gzip` in this field so that any records that are not-gzipped are rejected. none optional none Uncompressed. gzip optional gzip GZIP. store_access_key required boolean Whether or not to store the AWS Firehose Access Key in event secrets. If set to `true`, when incoming requests contains an access key sent by AWS Firehose, it is kept in the event secrets as "aws_kinesis_firehose_access_key". tls optional Configures the TLS options for incoming/outgoing connections. Option 1 optional Configures the TLS options for incoming/outgoing connections. enabled optional boolean,​null Whether or not to require TLS for incoming or outgoing connections. When enabled and used for incoming connections, an identity certificate is also required. See `tls.crt_file` for more information. alpn_protocols optional array,​null Sets the list of supported ALPN protocols. Declare the supported ALPN protocols, which are used during negotiation with peer. They are prioritized in the order that they are defined. ca_file optional Absolute path to an additional CA certificate file. The certificate must be in the DER or PEM (X.509) format. Additionally, the certificate can be provided as an inline string in PEM format. Option 1 optional string A file path. crt_file optional Absolute path to a certificate file used to identify this server. The certificate must be in DER, PEM (X.509), or PKCS#12 format. Additionally, the certificate can be provided as an inline string in PEM format. If this is set, and is not a PKCS#12 archive, `key_file` must also be set. Option 1 optional string A file path. key_file optional Absolute path to a private key file used to identify this server. The key must be in DER or PEM (PKCS#8) format. Additionally, the key can be provided as an inline string in PEM format. Option 1 optional string A file path. key_pass optional string,​null Passphrase used to unlock the encrypted key file. This has no effect unless `key_file` is set. verify_certificate optional boolean,​null Enables certificate verification. If enabled, certificates must not be expired and must be issued by a trusted issuer. This verification operates in a hierarchical manner, checking that the leaf certificate (the certificate presented by the client/server) is not only valid, but that the issuer of that certificate is also valid, and so on until the verification process reaches a root certificate. Relevant for both incoming and outgoing connections. Do NOT set this to `false` unless you understand the risks of not verifying the validity of certificates. verify_hostname optional boolean,​null Enables hostname verification. If enabled, the hostname used to connect to the remote host must be present in the TLS certificate presented by the remote host, either as the Common Name or as an entry in the Subject Alternative Name extension. Only relevant for outgoing connections. Do NOT set this to `false` unless you understand the risks of not verifying the remote hostname. {% /tab %} {% tab title="awskinesisfirehose-request-example" %} ```yaml access_key: '' access_keys: array acknowledgements: enabled: null address: string decoding: codec: bytes framing: method: bytes log_namespace: boolean record_compression: auto store_access_key: boolean tls: '' type: aws_kinesis_firehose ``` {% /tab %} ### AWS S3{% #awss3 %} Configuration for the `aws_s3` source. OptionsSchema {% tab title="awss3-request-model" %} FieldrequiredTypeDescription acknowledgements optional object Controls how acknowledgements are handled by this source. **DEPRECATED**: This setting is **deprecated** in favor of enabling `acknowledgements` at the [global](https://vector.dev/docs/reference/configuration/global-options/#acknowledgements) or sink level. Enabling or disabling acknowledgements at the source level has **no effect** on acknowledgement behavior. See [End-to-end Acknowledgements](https://vector.dev/docs/about/under-the-hood/architecture/end-to-end-acknowledgements/) for more information on how event acknowledgement is handled. enabled optional boolean,​null Whether or not end-to-end acknowledgements are enabled for this source. default: nullassume_role optional string,​null The ARN of an [IAM role][iam_role] to assume at startup. **DEPRECATED**: [iam_role]: [https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles.html](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles.html) auth optional Configuration of the authentication strategy for interacting with AWS services. compression optional The compression scheme used for decompressing objects retrieved from S3. auto optional auto Automatically attempt to determine the compression scheme. The compression scheme of the object is determined from its `Content-Encoding` and `Content-Type` metadata, as well as the key suffix (for example, `.gz`). It is set to `none` if the compression scheme cannot be determined. none optional none Uncompressed. gzip optional gzip GZIP. zstd optional zstd ZSTD. decoding optional Configures how events are decoded from raw bytes. Bytes optional object Uses the raw bytes as-is. codec required bytes Uses the raw bytes as-is. Json optional Decodes the raw bytes as [JSON](https://www.json.org/). json optional object JSON-specific decoding options. lossy optional boolean Determines whether or not to replace invalid UTF-8 sequences instead of failing. When true, invalid UTF-8 sequences are replaced with the [`U+FFFD REPLACEMENT CHARACTER`](https://en.wikipedia.org/wiki/Specials_\(Unicode_block\)#Replacement_character). default: truecodec required json Decodes the raw bytes as [JSON](https://www.json.org/). Protobuf optional Decodes the raw bytes as [protobuf](https://protobuf.dev/). protobuf optional object Protobuf-specific decoding options. desc_file required string Path to desc file message_type required string message type. e.g package.message codec required protobuf Decodes the raw bytes as [protobuf](https://protobuf.dev/). Syslog optional Decodes the raw bytes as a Syslog message. Decodes either as the [RFC 3164](https://www.ietf.org/rfc/rfc3164.txt)-style format ("old" style) or the [RFC 5424](https://www.ietf.org/rfc/rfc5424.txt)-style format ("new" style, includes structured data). syslog optional object Syslog-specific decoding options. lossy optional boolean Determines whether or not to replace invalid UTF-8 sequences instead of failing. When true, invalid UTF-8 sequences are replaced with the [`U+FFFD REPLACEMENT CHARACTER`](https://en.wikipedia.org/wiki/Specials_\(Unicode_block\)#Replacement_character). default: truecodec required syslog Decodes the raw bytes as a Syslog message. Decodes either as the [RFC 3164](https://www.ietf.org/rfc/rfc3164.txt)-style format ("old" style) or the [RFC 5424](https://www.ietf.org/rfc/rfc5424.txt)-style format ("new" style, includes structured data). Native optional object Decodes the raw bytes as Vector's [native Protocol Buffers format](https://github.com/vectordotdev/vector/blob/master/lib/vector-core/proto/event.proto). This codec is **[experimental](https://vector.dev/highlights/2022-03-31-native-event-codecs)**. codec required native Decodes the raw bytes as Vector's [native Protocol Buffers format](https://github.com/vectordotdev/vector/blob/master/lib/vector-core/proto/event.proto). This codec is **[experimental](https://vector.dev/highlights/2022-03-31-native-event-codecs)**. NativeJson optional Decodes the raw bytes as Vector's [native JSON format](https://github.com/vectordotdev/vector/blob/master/lib/codecs/tests/data/native_encoding/schema.cue). This codec is **[experimental](https://vector.dev/highlights/2022-03-31-native-event-codecs)**. native_json optional object Vector's native JSON-specific decoding options. lossy optional boolean Determines whether or not to replace invalid UTF-8 sequences instead of failing. When true, invalid UTF-8 sequences are replaced with the [`U+FFFD REPLACEMENT CHARACTER`](https://en.wikipedia.org/wiki/Specials_\(Unicode_block\)#Replacement_character). default: truecodec required native_json Decodes the raw bytes as Vector's [native JSON format](https://github.com/vectordotdev/vector/blob/master/lib/codecs/tests/data/native_encoding/schema.cue). This codec is **[experimental](https://vector.dev/highlights/2022-03-31-native-event-codecs)**. Gelf optional Decodes the raw bytes as a [GELF](https://docs.graylog.org/docs/gelf) message. gelf optional object GELF-specific decoding options. lossy optional boolean Determines whether or not to replace invalid UTF-8 sequences instead of failing. When true, invalid UTF-8 sequences are replaced with the [`U+FFFD REPLACEMENT CHARACTER`](https://en.wikipedia.org/wiki/Specials_\(Unicode_block\)#Replacement_character). default: truecodec required gelf Decodes the raw bytes as a [GELF](https://docs.graylog.org/docs/gelf) message. framing optional Framing configuration. Framing handles how events are separated when encoded in a raw byte form, where each event is a frame that must be prefixed, or delimited, in a way that marks where an event begins and ends within the byte stream. Bytes optional object Byte frames are passed through as-is according to the underlying I/O boundaries (for example, split between messages or stream segments). method required bytes Byte frames are passed through as-is according to the underlying I/O boundaries (for example, split between messages or stream segments). CharacterDelimited optional Byte frames which are delimited by a chosen character. character_delimited required object Options for the character delimited decoder. delimiter required integer The character that delimits byte sequences. max_length optional integer,​null The maximum length of the byte buffer. This length does *not* include the trailing delimiter. By default, there is no maximum length enforced. If events are malformed, this can lead to additional resource usage as events continue to be buffered in memory, and can potentially lead to memory exhaustion in extreme cases. If there is a risk of processing malformed data, such as logs with user-controlled input, consider setting the maximum length to a reasonably large value as a safety net. This ensures that processing is not actually unbounded. method required character_delimited Byte frames which are delimited by a chosen character. LengthDelimited optional object Byte frames which are prefixed by an unsigned big-endian 32-bit integer indicating the length. method required length_delimited Byte frames which are prefixed by an unsigned big-endian 32-bit integer indicating the length. NewlineDelimited optional Byte frames which are delimited by a newline character. newline_delimited optional object Options for the newline delimited decoder. max_length optional integer,​null The maximum length of the byte buffer. This length does *not* include the trailing delimiter. By default, there is no maximum length enforced. If events are malformed, this can lead to additional resource usage as events continue to be buffered in memory, and can potentially lead to memory exhaustion in extreme cases. If there is a risk of processing malformed data, such as logs with user-controlled input, consider setting the maximum length to a reasonably large value as a safety net. This ensures that processing is not actually unbounded. method required newline_delimited Byte frames which are delimited by a newline character. OctetCounting optional Byte frames according to the [octet counting](https://tools.ietf.org/html/rfc6587#section-3.4.1) format. octet_counting optional object Options for the octet counting decoder. max_length optional integer,​null The maximum length of the byte buffer. method required octet_counting Byte frames according to the [octet counting](https://tools.ietf.org/html/rfc6587#section-3.4.1) format. log_namespace optional boolean,​null The namespace to use for logs. This overrides the global setting. multiline optional Multiline aggregation configuration. If not specified, multiline aggregation is disabled. Option 1 optional object Configuration of multi-line aggregation. condition_pattern required string Regular expression pattern that is used to determine whether or not more lines should be read. This setting must be configured in conjunction with `mode`. mode required Aggregation mode. This setting must be configured in conjunction with `condition_pattern`. continue_through optional continue_through All consecutive lines matching this pattern are included in the group. The first line (the line that matched the start pattern) does not need to match the `ContinueThrough` pattern. This is useful in cases such as a Java stack trace, where some indicator in the line (such as a leading whitespace) indicates that it is an extension of the proceeding line. continue_past optional continue_past All consecutive lines matching this pattern, plus one additional line, are included in the group. This is useful in cases where a log message ends with a continuation marker, such as a backslash, indicating that the following line is part of the same message. halt_before optional halt_before All consecutive lines not matching this pattern are included in the group. This is useful where a log line contains a marker indicating that it begins a new message. halt_with optional halt_with All consecutive lines, up to and including the first line matching this pattern, are included in the group. This is useful where a log line ends with a termination marker, such as a semicolon. start_pattern required string Regular expression pattern that is used to match the start of a new message. timeout_ms required integer The maximum amount of time to wait for the next additional line, in milliseconds. Once this timeout is reached, the buffered message is guaranteed to be flushed, even if incomplete. sqs optional Configuration options for SQS. Option 1 optional object SQS configuration options. client_concurrency optional integer,​null Number of concurrent tasks to create for polling the queue for messages. Defaults to the number of available CPUs on the system. Should not typically need to be changed, but it can sometimes be beneficial to raise this value when there is a high rate of messages being pushed into the queue and the objects being fetched are small. In these cases, system resources may not be fully utilized without fetching more messages per second, as the SQS message consumption rate affects the S3 object retrieval rate. delete_message optional boolean Whether to delete the message once it is processed. It can be useful to set this to `false` for debugging or during the initial setup. poll_secs optional integer How long to wait while polling the queue for new messages, in seconds. Generally, this should not be changed unless instructed to do so, as if messages are available, they are always consumed, regardless of the value of `poll_secs`. queue_url required uri The URL of the SQS queue to poll for bucket notifications. tls_options optional TLS configuration. Option 1 optional object TLS configuration. alpn_protocols optional array,​null Sets the list of supported ALPN protocols. Declare the supported ALPN protocols, which are used during negotiation with peer. They are prioritized in the order that they are defined. ca_file optional Absolute path to an additional CA certificate file. The certificate must be in the DER or PEM (X.509) format. Additionally, the certificate can be provided as an inline string in PEM format. Option 1 optional string A file path. crt_file optional Absolute path to a certificate file used to identify this server. The certificate must be in DER, PEM (X.509), or PKCS#12 format. Additionally, the certificate can be provided as an inline string in PEM format. If this is set, and is not a PKCS#12 archive, `key_file` must also be set. Option 1 optional string A file path. key_file optional Absolute path to a private key file used to identify this server. The key must be in DER or PEM (PKCS#8) format. Additionally, the key can be provided as an inline string in PEM format. Option 1 optional string A file path. key_pass optional string,​null Passphrase used to unlock the encrypted key file. This has no effect unless `key_file` is set. verify_certificate optional boolean,​null Enables certificate verification. If enabled, certificates must not be expired and must be issued by a trusted issuer. This verification operates in a hierarchical manner, checking that the leaf certificate (the certificate presented by the client/server) is not only valid, but that the issuer of that certificate is also valid, and so on until the verification process reaches a root certificate. Relevant for both incoming and outgoing connections. Do NOT set this to `false` unless you understand the risks of not verifying the validity of certificates. verify_hostname optional boolean,​null Enables hostname verification. If enabled, the hostname used to connect to the remote host must be present in the TLS certificate presented by the remote host, either as the Common Name or as an entry in the Subject Alternative Name extension. Only relevant for outgoing connections. Do NOT set this to `false` unless you understand the risks of not verifying the remote hostname. visibility_timeout_secs optional integer The visibility timeout to use for messages, in seconds. This controls how long a message is left unavailable after it is received. If a message is received, and takes longer than `visibility_timeout_secs` to process and delete the message from the queue, it is made available again for another consumer. This can happen if there is an issue between consuming a message and deleting it. strategy optional The strategy to use to consume objects from S3. sqs optional sqs Consumes objects by processing bucket notification events sent to an [AWS SQS queue](https://aws.amazon.com/sqs/). tls_options optional TLS configuration. Option 1 optional object TLS configuration. alpn_protocols optional array,​null Sets the list of supported ALPN protocols. Declare the supported ALPN protocols, which are used during negotiation with peer. They are prioritized in the order that they are defined. ca_file optional Absolute path to an additional CA certificate file. The certificate must be in the DER or PEM (X.509) format. Additionally, the certificate can be provided as an inline string in PEM format. Option 1 optional string A file path. crt_file optional Absolute path to a certificate file used to identify this server. The certificate must be in DER, PEM (X.509), or PKCS#12 format. Additionally, the certificate can be provided as an inline string in PEM format. If this is set, and is not a PKCS#12 archive, `key_file` must also be set. Option 1 optional string A file path. key_file optional Absolute path to a private key file used to identify this server. The key must be in DER or PEM (PKCS#8) format. Additionally, the key can be provided as an inline string in PEM format. Option 1 optional string A file path. key_pass optional string,​null Passphrase used to unlock the encrypted key file. This has no effect unless `key_file` is set. verify_certificate optional boolean,​null Enables certificate verification. If enabled, certificates must not be expired and must be issued by a trusted issuer. This verification operates in a hierarchical manner, checking that the leaf certificate (the certificate presented by the client/server) is not only valid, but that the issuer of that certificate is also valid, and so on until the verification process reaches a root certificate. Relevant for both incoming and outgoing connections. Do NOT set this to `false` unless you understand the risks of not verifying the validity of certificates. verify_hostname optional boolean,​null Enables hostname verification. If enabled, the hostname used to connect to the remote host must be present in the TLS certificate presented by the remote host, either as the Common Name or as an entry in the Subject Alternative Name extension. Only relevant for outgoing connections. Do NOT set this to `false` unless you understand the risks of not verifying the remote hostname. endpoint optional string,​null Custom endpoint for use with AWS-compatible services. region optional string,​null The [AWS region](https://docs.aws.amazon.com/general/latest/gr/rande.html#regional-endpoints) of the target service. {% /tab %} {% tab title="awss3-request-example" %} ```yaml acknowledgements: enabled: null assume_role: string auth: imds: connect_timeout_seconds: 1 max_attempts: 4 read_timeout_seconds: 1 load_timeout_secs: null region: null compression: '' decoding: codec: bytes framing: method: newline_delimited log_namespace: boolean multiline: '' sqs: '' strategy: '' tls_options: '' type: aws_s3 ``` {% /tab %} ### AWS SQS{% #awssqs %} Configuration for the `aws_sqs` source. OptionsSchema {% tab title="awssqs-request-model" %} FieldrequiredTypeDescription acknowledgements optional object Controls how acknowledgements are handled by this source. **DEPRECATED**: This setting is **deprecated** in favor of enabling `acknowledgements` at the [global](https://vector.dev/docs/reference/configuration/global-options/#acknowledgements) or sink level. Enabling or disabling acknowledgements at the source level has **no effect** on acknowledgement behavior. See [End-to-end Acknowledgements](https://vector.dev/docs/about/under-the-hood/architecture/end-to-end-acknowledgements/) for more information on how event acknowledgement is handled. enabled optional boolean,​null Whether or not end-to-end acknowledgements are enabled for this source. default: nullauth optional Configuration of the authentication strategy for interacting with AWS services. client_concurrency optional integer,​null Number of concurrent tasks to create for polling the queue for messages. Defaults to the number of available CPUs on the system. Should not typically need to be changed, but it can sometimes be beneficial to raise this value when there is a high rate of messages being pushed into the queue and the messages being fetched are small. In these cases, system resources may not be fully utilized without fetching more messages per second, as it spends more time fetching the messages than processing them. decoding optional Configures how events are decoded from raw bytes. Bytes optional object Uses the raw bytes as-is. codec required bytes Uses the raw bytes as-is. Json optional Decodes the raw bytes as [JSON](https://www.json.org/). json optional object JSON-specific decoding options. lossy optional boolean Determines whether or not to replace invalid UTF-8 sequences instead of failing. When true, invalid UTF-8 sequences are replaced with the [`U+FFFD REPLACEMENT CHARACTER`](https://en.wikipedia.org/wiki/Specials_\(Unicode_block\)#Replacement_character). default: truecodec required json Decodes the raw bytes as [JSON](https://www.json.org/). Protobuf optional Decodes the raw bytes as [protobuf](https://protobuf.dev/). protobuf optional object Protobuf-specific decoding options. desc_file required string Path to desc file message_type required string message type. e.g package.message codec required protobuf Decodes the raw bytes as [protobuf](https://protobuf.dev/). Syslog optional Decodes the raw bytes as a Syslog message. Decodes either as the [RFC 3164](https://www.ietf.org/rfc/rfc3164.txt)-style format ("old" style) or the [RFC 5424](https://www.ietf.org/rfc/rfc5424.txt)-style format ("new" style, includes structured data). syslog optional object Syslog-specific decoding options. lossy optional boolean Determines whether or not to replace invalid UTF-8 sequences instead of failing. When true, invalid UTF-8 sequences are replaced with the [`U+FFFD REPLACEMENT CHARACTER`](https://en.wikipedia.org/wiki/Specials_\(Unicode_block\)#Replacement_character). default: truecodec required syslog Decodes the raw bytes as a Syslog message. Decodes either as the [RFC 3164](https://www.ietf.org/rfc/rfc3164.txt)-style format ("old" style) or the [RFC 5424](https://www.ietf.org/rfc/rfc5424.txt)-style format ("new" style, includes structured data). Native optional object Decodes the raw bytes as Vector's [native Protocol Buffers format](https://github.com/vectordotdev/vector/blob/master/lib/vector-core/proto/event.proto). This codec is **[experimental](https://vector.dev/highlights/2022-03-31-native-event-codecs)**. codec required native Decodes the raw bytes as Vector's [native Protocol Buffers format](https://github.com/vectordotdev/vector/blob/master/lib/vector-core/proto/event.proto). This codec is **[experimental](https://vector.dev/highlights/2022-03-31-native-event-codecs)**. NativeJson optional Decodes the raw bytes as Vector's [native JSON format](https://github.com/vectordotdev/vector/blob/master/lib/codecs/tests/data/native_encoding/schema.cue). This codec is **[experimental](https://vector.dev/highlights/2022-03-31-native-event-codecs)**. native_json optional object Vector's native JSON-specific decoding options. lossy optional boolean Determines whether or not to replace invalid UTF-8 sequences instead of failing. When true, invalid UTF-8 sequences are replaced with the [`U+FFFD REPLACEMENT CHARACTER`](https://en.wikipedia.org/wiki/Specials_\(Unicode_block\)#Replacement_character). default: truecodec required native_json Decodes the raw bytes as Vector's [native JSON format](https://github.com/vectordotdev/vector/blob/master/lib/codecs/tests/data/native_encoding/schema.cue). This codec is **[experimental](https://vector.dev/highlights/2022-03-31-native-event-codecs)**. Gelf optional Decodes the raw bytes as a [GELF](https://docs.graylog.org/docs/gelf) message. gelf optional object GELF-specific decoding options. lossy optional boolean Determines whether or not to replace invalid UTF-8 sequences instead of failing. When true, invalid UTF-8 sequences are replaced with the [`U+FFFD REPLACEMENT CHARACTER`](https://en.wikipedia.org/wiki/Specials_\(Unicode_block\)#Replacement_character). default: truecodec required gelf Decodes the raw bytes as a [GELF](https://docs.graylog.org/docs/gelf) message. delete_message optional boolean Whether to delete the message once it is processed. It can be useful to set this to `false` for debugging or during the initial setup. framing optional Framing configuration. Framing handles how events are separated when encoded in a raw byte form, where each event is a frame that must be prefixed, or delimited, in a way that marks where an event begins and ends within the byte stream. Bytes optional object Byte frames are passed through as-is according to the underlying I/O boundaries (for example, split between messages or stream segments). method required bytes Byte frames are passed through as-is according to the underlying I/O boundaries (for example, split between messages or stream segments). CharacterDelimited optional Byte frames which are delimited by a chosen character. character_delimited required object Options for the character delimited decoder. delimiter required integer The character that delimits byte sequences. max_length optional integer,​null The maximum length of the byte buffer. This length does *not* include the trailing delimiter. By default, there is no maximum length enforced. If events are malformed, this can lead to additional resource usage as events continue to be buffered in memory, and can potentially lead to memory exhaustion in extreme cases. If there is a risk of processing malformed data, such as logs with user-controlled input, consider setting the maximum length to a reasonably large value as a safety net. This ensures that processing is not actually unbounded. method required character_delimited Byte frames which are delimited by a chosen character. LengthDelimited optional object Byte frames which are prefixed by an unsigned big-endian 32-bit integer indicating the length. method required length_delimited Byte frames which are prefixed by an unsigned big-endian 32-bit integer indicating the length. NewlineDelimited optional Byte frames which are delimited by a newline character. newline_delimited optional object Options for the newline delimited decoder. max_length optional integer,​null The maximum length of the byte buffer. This length does *not* include the trailing delimiter. By default, there is no maximum length enforced. If events are malformed, this can lead to additional resource usage as events continue to be buffered in memory, and can potentially lead to memory exhaustion in extreme cases. If there is a risk of processing malformed data, such as logs with user-controlled input, consider setting the maximum length to a reasonably large value as a safety net. This ensures that processing is not actually unbounded. method required newline_delimited Byte frames which are delimited by a newline character. OctetCounting optional Byte frames according to the [octet counting](https://tools.ietf.org/html/rfc6587#section-3.4.1) format. octet_counting optional object Options for the octet counting decoder. max_length optional integer,​null The maximum length of the byte buffer. method required octet_counting Byte frames according to the [octet counting](https://tools.ietf.org/html/rfc6587#section-3.4.1) format. log_namespace optional boolean,​null The namespace to use for logs. This overrides the global setting. poll_secs optional integer How long to wait while polling the queue for new messages, in seconds. Generally, this should not be changed unless instructed to do so, as if messages are available, they are always consumed, regardless of the value of `poll_secs`. queue_url required string The URL of the SQS queue to poll for messages. tls optional TLS configuration. Option 1 optional object TLS configuration. alpn_protocols optional array,​null Sets the list of supported ALPN protocols. Declare the supported ALPN protocols, which are used during negotiation with peer. They are prioritized in the order that they are defined. ca_file optional Absolute path to an additional CA certificate file. The certificate must be in the DER or PEM (X.509) format. Additionally, the certificate can be provided as an inline string in PEM format. Option 1 optional string A file path. crt_file optional Absolute path to a certificate file used to identify this server. The certificate must be in DER, PEM (X.509), or PKCS#12 format. Additionally, the certificate can be provided as an inline string in PEM format. If this is set, and is not a PKCS#12 archive, `key_file` must also be set. Option 1 optional string A file path. key_file optional Absolute path to a private key file used to identify this server. The key must be in DER or PEM (PKCS#8) format. Additionally, the key can be provided as an inline string in PEM format. Option 1 optional string A file path. key_pass optional string,​null Passphrase used to unlock the encrypted key file. This has no effect unless `key_file` is set. verify_certificate optional boolean,​null Enables certificate verification. If enabled, certificates must not be expired and must be issued by a trusted issuer. This verification operates in a hierarchical manner, checking that the leaf certificate (the certificate presented by the client/server) is not only valid, but that the issuer of that certificate is also valid, and so on until the verification process reaches a root certificate. Relevant for both incoming and outgoing connections. Do NOT set this to `false` unless you understand the risks of not verifying the validity of certificates. verify_hostname optional boolean,​null Enables hostname verification. If enabled, the hostname used to connect to the remote host must be present in the TLS certificate presented by the remote host, either as the Common Name or as an entry in the Subject Alternative Name extension. Only relevant for outgoing connections. Do NOT set this to `false` unless you understand the risks of not verifying the remote hostname. visibility_timeout_secs optional integer The visibility timeout to use for messages, in seconds. This controls how long a message is left unavailable after it is received. If a message is received, and takes longer than `visibility_timeout_secs` to process and delete the message from the queue, it is made available again for another consumer. This can happen if there is an issue between consuming a message and deleting it. endpoint optional string,​null Custom endpoint for use with AWS-compatible services. region optional string,​null The [AWS region](https://docs.aws.amazon.com/general/latest/gr/rande.html#regional-endpoints) of the target service. {% /tab %} {% tab title="awssqs-request-example" %} ```yaml acknowledgements: enabled: null auth: imds: connect_timeout_seconds: 1 max_attempts: 4 read_timeout_seconds: 1 load_timeout_secs: null region: null client_concurrency: integer decoding: codec: bytes delete_message: true framing: method: bytes log_namespace: boolean poll_secs: 15 queue_url: string tls: '' visibility_timeout_secs: 300 type: aws_sqs ``` {% /tab %} ### Datadog Agent{% #datadogagent %} Configuration for the `datadog_agent` source. OptionsSchema {% tab title="datadogagent-request-model" %} FieldrequiredTypeDescription acknowledgements optional object Controls how acknowledgements are handled by this source. **DEPRECATED**: This setting is **deprecated** in favor of enabling `acknowledgements` at the [global](https://vector.dev/docs/reference/configuration/global-options/#acknowledgements) or sink level. Enabling or disabling acknowledgements at the source level has **no effect** on acknowledgement behavior. See [End-to-end Acknowledgements](https://vector.dev/docs/about/under-the-hood/architecture/end-to-end-acknowledgements/) for more information on how event acknowledgement is handled. enabled optional boolean,​null Whether or not end-to-end acknowledgements are enabled for this source. default: nulladdress required string The socket address to accept connections on. It *must* include a port. decoding optional Configures how events are decoded from raw bytes. Bytes optional object Uses the raw bytes as-is. codec required bytes Uses the raw bytes as-is. Json optional Decodes the raw bytes as [JSON](https://www.json.org/). json optional object JSON-specific decoding options. lossy optional boolean Determines whether or not to replace invalid UTF-8 sequences instead of failing. When true, invalid UTF-8 sequences are replaced with the [`U+FFFD REPLACEMENT CHARACTER`](https://en.wikipedia.org/wiki/Specials_\(Unicode_block\)#Replacement_character). default: truecodec required json Decodes the raw bytes as [JSON](https://www.json.org/). Protobuf optional Decodes the raw bytes as [protobuf](https://protobuf.dev/). protobuf optional object Protobuf-specific decoding options. desc_file required string Path to desc file message_type required string message type. e.g package.message codec required protobuf Decodes the raw bytes as [protobuf](https://protobuf.dev/). Syslog optional Decodes the raw bytes as a Syslog message. Decodes either as the [RFC 3164](https://www.ietf.org/rfc/rfc3164.txt)-style format ("old" style) or the [RFC 5424](https://www.ietf.org/rfc/rfc5424.txt)-style format ("new" style, includes structured data). syslog optional object Syslog-specific decoding options. lossy optional boolean Determines whether or not to replace invalid UTF-8 sequences instead of failing. When true, invalid UTF-8 sequences are replaced with the [`U+FFFD REPLACEMENT CHARACTER`](https://en.wikipedia.org/wiki/Specials_\(Unicode_block\)#Replacement_character). default: truecodec required syslog Decodes the raw bytes as a Syslog message. Decodes either as the [RFC 3164](https://www.ietf.org/rfc/rfc3164.txt)-style format ("old" style) or the [RFC 5424](https://www.ietf.org/rfc/rfc5424.txt)-style format ("new" style, includes structured data). Native optional object Decodes the raw bytes as Vector's [native Protocol Buffers format](https://github.com/vectordotdev/vector/blob/master/lib/vector-core/proto/event.proto). This codec is **[experimental](https://vector.dev/highlights/2022-03-31-native-event-codecs)**. codec required native Decodes the raw bytes as Vector's [native Protocol Buffers format](https://github.com/vectordotdev/vector/blob/master/lib/vector-core/proto/event.proto). This codec is **[experimental](https://vector.dev/highlights/2022-03-31-native-event-codecs)**. NativeJson optional Decodes the raw bytes as Vector's [native JSON format](https://github.com/vectordotdev/vector/blob/master/lib/codecs/tests/data/native_encoding/schema.cue). This codec is **[experimental](https://vector.dev/highlights/2022-03-31-native-event-codecs)**. native_json optional object Vector's native JSON-specific decoding options. lossy optional boolean Determines whether or not to replace invalid UTF-8 sequences instead of failing. When true, invalid UTF-8 sequences are replaced with the [`U+FFFD REPLACEMENT CHARACTER`](https://en.wikipedia.org/wiki/Specials_\(Unicode_block\)#Replacement_character). default: truecodec required native_json Decodes the raw bytes as Vector's [native JSON format](https://github.com/vectordotdev/vector/blob/master/lib/codecs/tests/data/native_encoding/schema.cue). This codec is **[experimental](https://vector.dev/highlights/2022-03-31-native-event-codecs)**. Gelf optional Decodes the raw bytes as a [GELF](https://docs.graylog.org/docs/gelf) message. gelf optional object GELF-specific decoding options. lossy optional boolean Determines whether or not to replace invalid UTF-8 sequences instead of failing. When true, invalid UTF-8 sequences are replaced with the [`U+FFFD REPLACEMENT CHARACTER`](https://en.wikipedia.org/wiki/Specials_\(Unicode_block\)#Replacement_character). default: truecodec required gelf Decodes the raw bytes as a [GELF](https://docs.graylog.org/docs/gelf) message. disable_logs optional boolean If this is set to `true`, logs are not accepted by the component. disable_metrics optional boolean If this is set to `true`, metrics are not accepted by the component. disable_traces optional boolean If this is set to `true`, traces are not accepted by the component. framing optional Framing configuration. Framing handles how events are separated when encoded in a raw byte form, where each event is a frame that must be prefixed, or delimited, in a way that marks where an event begins and ends within the byte stream. Bytes optional object Byte frames are passed through as-is according to the underlying I/O boundaries (for example, split between messages or stream segments). method required bytes Byte frames are passed through as-is according to the underlying I/O boundaries (for example, split between messages or stream segments). CharacterDelimited optional Byte frames which are delimited by a chosen character. character_delimited required object Options for the character delimited decoder. delimiter required integer The character that delimits byte sequences. max_length optional integer,​null The maximum length of the byte buffer. This length does *not* include the trailing delimiter. By default, there is no maximum length enforced. If events are malformed, this can lead to additional resource usage as events continue to be buffered in memory, and can potentially lead to memory exhaustion in extreme cases. If there is a risk of processing malformed data, such as logs with user-controlled input, consider setting the maximum length to a reasonably large value as a safety net. This ensures that processing is not actually unbounded. method required character_delimited Byte frames which are delimited by a chosen character. LengthDelimited optional object Byte frames which are prefixed by an unsigned big-endian 32-bit integer indicating the length. method required length_delimited Byte frames which are prefixed by an unsigned big-endian 32-bit integer indicating the length. NewlineDelimited optional Byte frames which are delimited by a newline character. newline_delimited optional object Options for the newline delimited decoder. max_length optional integer,​null The maximum length of the byte buffer. This length does *not* include the trailing delimiter. By default, there is no maximum length enforced. If events are malformed, this can lead to additional resource usage as events continue to be buffered in memory, and can potentially lead to memory exhaustion in extreme cases. If there is a risk of processing malformed data, such as logs with user-controlled input, consider setting the maximum length to a reasonably large value as a safety net. This ensures that processing is not actually unbounded. method required newline_delimited Byte frames which are delimited by a newline character. OctetCounting optional Byte frames according to the [octet counting](https://tools.ietf.org/html/rfc6587#section-3.4.1) format. octet_counting optional object Options for the octet counting decoder. max_length optional integer,​null The maximum length of the byte buffer. method required octet_counting Byte frames according to the [octet counting](https://tools.ietf.org/html/rfc6587#section-3.4.1) format. log_namespace optional boolean,​null The namespace to use for logs. This overrides the global setting. multiple_outputs optional boolean If this is set to `true` logs, metrics, and traces are sent to different outputs. For a source component named `agent`, the received logs, metrics, and traces can then be configured as input to other components by specifying `agent.logs`, `agent.metrics`, and `agent.traces`, respectively. store_api_key optional boolean If this is set to `true`, when incoming events contain a Datadog API key, it is stored in the event metadata and used if the event is sent to a Datadog sink. tls optional Configures the TLS options for incoming/outgoing connections. Option 1 optional Configures the TLS options for incoming/outgoing connections. enabled optional boolean,​null Whether or not to require TLS for incoming or outgoing connections. When enabled and used for incoming connections, an identity certificate is also required. See `tls.crt_file` for more information. alpn_protocols optional array,​null Sets the list of supported ALPN protocols. Declare the supported ALPN protocols, which are used during negotiation with peer. They are prioritized in the order that they are defined. ca_file optional Absolute path to an additional CA certificate file. The certificate must be in the DER or PEM (X.509) format. Additionally, the certificate can be provided as an inline string in PEM format. Option 1 optional string A file path. crt_file optional Absolute path to a certificate file used to identify this server. The certificate must be in DER, PEM (X.509), or PKCS#12 format. Additionally, the certificate can be provided as an inline string in PEM format. If this is set, and is not a PKCS#12 archive, `key_file` must also be set. Option 1 optional string A file path. key_file optional Absolute path to a private key file used to identify this server. The key must be in DER or PEM (PKCS#8) format. Additionally, the key can be provided as an inline string in PEM format. Option 1 optional string A file path. key_pass optional string,​null Passphrase used to unlock the encrypted key file. This has no effect unless `key_file` is set. verify_certificate optional boolean,​null Enables certificate verification. If enabled, certificates must not be expired and must be issued by a trusted issuer. This verification operates in a hierarchical manner, checking that the leaf certificate (the certificate presented by the client/server) is not only valid, but that the issuer of that certificate is also valid, and so on until the verification process reaches a root certificate. Relevant for both incoming and outgoing connections. Do NOT set this to `false` unless you understand the risks of not verifying the validity of certificates. verify_hostname optional boolean,​null Enables hostname verification. If enabled, the hostname used to connect to the remote host must be present in the TLS certificate presented by the remote host, either as the Common Name or as an entry in the Subject Alternative Name extension. Only relevant for outgoing connections. Do NOT set this to `false` unless you understand the risks of not verifying the remote hostname. {% /tab %} {% tab title="datadogagent-request-example" %} ```yaml acknowledgements: enabled: null address: string decoding: codec: bytes disable_logs: boolean disable_metrics: boolean disable_traces: boolean framing: method: bytes log_namespace: boolean multiple_outputs: boolean store_api_key: true tls: '' type: datadog_agent ``` {% /tab %} ### Demo Logs{% #demologs %} Configuration for the `demo_logs` source. OptionsSchema {% tab title="demologs-request-model" %} FieldrequiredTypeDescriptioncount optional integer The total number of lines to output. By default, the source continuously prints logs (infinitely). decoding optional Configures how events are decoded from raw bytes. Bytes optional object Uses the raw bytes as-is. codec required bytes Uses the raw bytes as-is. Json optional Decodes the raw bytes as [JSON](https://www.json.org/). json optional object JSON-specific decoding options. lossy optional boolean Determines whether or not to replace invalid UTF-8 sequences instead of failing. When true, invalid UTF-8 sequences are replaced with the [`U+FFFD REPLACEMENT CHARACTER`](https://en.wikipedia.org/wiki/Specials_\(Unicode_block\)#Replacement_character). default: truecodec required json Decodes the raw bytes as [JSON](https://www.json.org/). Protobuf optional Decodes the raw bytes as [protobuf](https://protobuf.dev/). protobuf optional object Protobuf-specific decoding options. desc_file required string Path to desc file message_type required string message type. e.g package.message codec required protobuf Decodes the raw bytes as [protobuf](https://protobuf.dev/). Syslog optional Decodes the raw bytes as a Syslog message. Decodes either as the [RFC 3164](https://www.ietf.org/rfc/rfc3164.txt)-style format ("old" style) or the [RFC 5424](https://www.ietf.org/rfc/rfc5424.txt)-style format ("new" style, includes structured data). syslog optional object Syslog-specific decoding options. lossy optional boolean Determines whether or not to replace invalid UTF-8 sequences instead of failing. When true, invalid UTF-8 sequences are replaced with the [`U+FFFD REPLACEMENT CHARACTER`](https://en.wikipedia.org/wiki/Specials_\(Unicode_block\)#Replacement_character). default: truecodec required syslog Decodes the raw bytes as a Syslog message. Decodes either as the [RFC 3164](https://www.ietf.org/rfc/rfc3164.txt)-style format ("old" style) or the [RFC 5424](https://www.ietf.org/rfc/rfc5424.txt)-style format ("new" style, includes structured data). Native optional object Decodes the raw bytes as Vector's [native Protocol Buffers format](https://github.com/vectordotdev/vector/blob/master/lib/vector-core/proto/event.proto). This codec is **[experimental](https://vector.dev/highlights/2022-03-31-native-event-codecs)**. codec required native Decodes the raw bytes as Vector's [native Protocol Buffers format](https://github.com/vectordotdev/vector/blob/master/lib/vector-core/proto/event.proto). This codec is **[experimental](https://vector.dev/highlights/2022-03-31-native-event-codecs)**. NativeJson optional Decodes the raw bytes as Vector's [native JSON format](https://github.com/vectordotdev/vector/blob/master/lib/codecs/tests/data/native_encoding/schema.cue). This codec is **[experimental](https://vector.dev/highlights/2022-03-31-native-event-codecs)**. native_json optional object Vector's native JSON-specific decoding options. lossy optional boolean Determines whether or not to replace invalid UTF-8 sequences instead of failing. When true, invalid UTF-8 sequences are replaced with the [`U+FFFD REPLACEMENT CHARACTER`](https://en.wikipedia.org/wiki/Specials_\(Unicode_block\)#Replacement_character). default: truecodec required native_json Decodes the raw bytes as Vector's [native JSON format](https://github.com/vectordotdev/vector/blob/master/lib/codecs/tests/data/native_encoding/schema.cue). This codec is **[experimental](https://vector.dev/highlights/2022-03-31-native-event-codecs)**. Gelf optional Decodes the raw bytes as a [GELF](https://docs.graylog.org/docs/gelf) message. gelf optional object GELF-specific decoding options. lossy optional boolean Determines whether or not to replace invalid UTF-8 sequences instead of failing. When true, invalid UTF-8 sequences are replaced with the [`U+FFFD REPLACEMENT CHARACTER`](https://en.wikipedia.org/wiki/Specials_\(Unicode_block\)#Replacement_character). default: truecodec required gelf Decodes the raw bytes as a [GELF](https://docs.graylog.org/docs/gelf) message. framing optional Framing configuration. Framing handles how events are separated when encoded in a raw byte form, where each event is a frame that must be prefixed, or delimited, in a way that marks where an event begins and ends within the byte stream. Bytes optional object Byte frames are passed through as-is according to the underlying I/O boundaries (for example, split between messages or stream segments). method required bytes Byte frames are passed through as-is according to the underlying I/O boundaries (for example, split between messages or stream segments). CharacterDelimited optional Byte frames which are delimited by a chosen character. character_delimited required object Options for the character delimited decoder. delimiter required integer The character that delimits byte sequences. max_length optional integer,​null The maximum length of the byte buffer. This length does *not* include the trailing delimiter. By default, there is no maximum length enforced. If events are malformed, this can lead to additional resource usage as events continue to be buffered in memory, and can potentially lead to memory exhaustion in extreme cases. If there is a risk of processing malformed data, such as logs with user-controlled input, consider setting the maximum length to a reasonably large value as a safety net. This ensures that processing is not actually unbounded. method required character_delimited Byte frames which are delimited by a chosen character. LengthDelimited optional object Byte frames which are prefixed by an unsigned big-endian 32-bit integer indicating the length. method required length_delimited Byte frames which are prefixed by an unsigned big-endian 32-bit integer indicating the length. NewlineDelimited optional Byte frames which are delimited by a newline character. newline_delimited optional object Options for the newline delimited decoder. max_length optional integer,​null The maximum length of the byte buffer. This length does *not* include the trailing delimiter. By default, there is no maximum length enforced. If events are malformed, this can lead to additional resource usage as events continue to be buffered in memory, and can potentially lead to memory exhaustion in extreme cases. If there is a risk of processing malformed data, such as logs with user-controlled input, consider setting the maximum length to a reasonably large value as a safety net. This ensures that processing is not actually unbounded. method required newline_delimited Byte frames which are delimited by a newline character. OctetCounting optional Byte frames according to the [octet counting](https://tools.ietf.org/html/rfc6587#section-3.4.1) format. octet_counting optional object Options for the octet counting decoder. max_length optional integer,​null The maximum length of the byte buffer. method required octet_counting Byte frames according to the [octet counting](https://tools.ietf.org/html/rfc6587#section-3.4.1) format. interval optional number The amount of time, in seconds, to pause between each batch of output lines. The default is one batch per second. To remove the delay and output batches as quickly as possible, set `interval` to `0.0`. log_namespace optional boolean,​null The namespace to use for logs. This overrides the global setting. _metadata optional description optional Output format configuration. oneOf optional {% /tab %} {% tab title="demologs-request-example" %} ```yaml count: 9223372036854776000 decoding: codec: bytes framing: method: bytes interval: 1 log_namespace: boolean type: demo_logs ``` {% /tab %} ### Fluent{% #fluent %} Configuration for the `fluent` source. OptionsSchema {% tab title="fluent-request-model" %} FieldrequiredTypeDescription acknowledgements optional object Controls how acknowledgements are handled by this source. **DEPRECATED**: This setting is **deprecated** in favor of enabling `acknowledgements` at the [global](https://vector.dev/docs/reference/configuration/global-options/#acknowledgements) or sink level. Enabling or disabling acknowledgements at the source level has **no effect** on acknowledgement behavior. See [End-to-end Acknowledgements](https://vector.dev/docs/about/under-the-hood/architecture/end-to-end-acknowledgements/) for more information on how event acknowledgement is handled. enabled optional boolean,​null Whether or not end-to-end acknowledgements are enabled for this source. default: nulladdress required string The socket address to listen for connections on, or `systemd{#N}` to use the Nth socket passed by systemd socket activation. If a socket address is used, it *must* include a port. connection_limit optional integer,​null The maximum number of TCP connections that are allowed at any given time. keepalive optional TCP keepalive settings for socket-based components. Option 1 optional object TCP keepalive settings for socket-based components. time_secs optional integer,​null The time to wait before starting to send TCP keepalive probes on an idle connection. log_namespace optional boolean,​null The namespace to use for logs. This overrides the global setting. receive_buffer_bytes optional integer,​null The size of the receive buffer used for each connection. This generally should not need to be changed. tls optional TlsEnableableConfig for `sources`, adding metadata from the client certificate. Option 1 optional TlsEnableableConfig for `sources`, adding metadata from the client certificate. client_metadata_key optional Event field for client certificate metadata. Option 1 optional string An optional path that deserializes an empty string to `None`. enabled optional boolean,​null Whether or not to require TLS for incoming or outgoing connections. When enabled and used for incoming connections, an identity certificate is also required. See `tls.crt_file` for more information. alpn_protocols optional array,​null Sets the list of supported ALPN protocols. Declare the supported ALPN protocols, which are used during negotiation with peer. They are prioritized in the order that they are defined. ca_file optional Absolute path to an additional CA certificate file. The certificate must be in the DER or PEM (X.509) format. Additionally, the certificate can be provided as an inline string in PEM format. Option 1 optional string A file path. crt_file optional Absolute path to a certificate file used to identify this server. The certificate must be in DER, PEM (X.509), or PKCS#12 format. Additionally, the certificate can be provided as an inline string in PEM format. If this is set, and is not a PKCS#12 archive, `key_file` must also be set. Option 1 optional string A file path. key_file optional Absolute path to a private key file used to identify this server. The key must be in DER or PEM (PKCS#8) format. Additionally, the key can be provided as an inline string in PEM format. Option 1 optional string A file path. key_pass optional string,​null Passphrase used to unlock the encrypted key file. This has no effect unless `key_file` is set. verify_certificate optional boolean,​null Enables certificate verification. If enabled, certificates must not be expired and must be issued by a trusted issuer. This verification operates in a hierarchical manner, checking that the leaf certificate (the certificate presented by the client/server) is not only valid, but that the issuer of that certificate is also valid, and so on until the verification process reaches a root certificate. Relevant for both incoming and outgoing connections. Do NOT set this to `false` unless you understand the risks of not verifying the validity of certificates. verify_hostname optional boolean,​null Enables hostname verification. If enabled, the hostname used to connect to the remote host must be present in the TLS certificate presented by the remote host, either as the Common Name or as an entry in the Subject Alternative Name extension. Only relevant for outgoing connections. Do NOT set this to `false` unless you understand the risks of not verifying the remote hostname. {% /tab %} {% tab title="fluent-request-example" %} ```yaml acknowledgements: enabled: null address: string connection_limit: integer keepalive: '' log_namespace: boolean receive_buffer_bytes: integer tls: '' type: fluent ``` {% /tab %} ### GCP Pub/Sub{% #pubsub %} Configuration for the `gcp_pubsub` source. OptionsSchema {% tab title="pubsub-request-model" %} FieldrequiredTypeDescriptionack_deadline_seconds optional integer,​null The acknowledgement deadline, in seconds, to use for this stream. **DEPRECATED**: Messages that are not acknowledged when this deadline expires may be retransmitted. ack_deadline_secs optional integer The acknowledgement deadline, in seconds, to use for this stream. Messages that are not acknowledged when this deadline expires may be retransmitted. acknowledgements optional object Controls how acknowledgements are handled by this source. **DEPRECATED**: This setting is **deprecated** in favor of enabling `acknowledgements` at the [global](https://vector.dev/docs/reference/configuration/global-options/#acknowledgements) or sink level. Enabling or disabling acknowledgements at the source level has **no effect** on acknowledgement behavior. See [End-to-end Acknowledgements](https://vector.dev/docs/about/under-the-hood/architecture/end-to-end-acknowledgements/) for more information on how event acknowledgement is handled. enabled optional boolean,​null Whether or not end-to-end acknowledgements are enabled for this source. default: null decoding optional Configures how events are decoded from raw bytes. Bytes optional object Uses the raw bytes as-is. codec required bytes Uses the raw bytes as-is. Json optional Decodes the raw bytes as [JSON](https://www.json.org/). json optional object JSON-specific decoding options. lossy optional boolean Determines whether or not to replace invalid UTF-8 sequences instead of failing. When true, invalid UTF-8 sequences are replaced with the [`U+FFFD REPLACEMENT CHARACTER`](https://en.wikipedia.org/wiki/Specials_\(Unicode_block\)#Replacement_character). default: truecodec required json Decodes the raw bytes as [JSON](https://www.json.org/). Protobuf optional Decodes the raw bytes as [protobuf](https://protobuf.dev/). protobuf optional object Protobuf-specific decoding options. desc_file required string Path to desc file message_type required string message type. e.g package.message codec required protobuf Decodes the raw bytes as [protobuf](https://protobuf.dev/). Syslog optional Decodes the raw bytes as a Syslog message. Decodes either as the [RFC 3164](https://www.ietf.org/rfc/rfc3164.txt)-style format ("old" style) or the [RFC 5424](https://www.ietf.org/rfc/rfc5424.txt)-style format ("new" style, includes structured data). syslog optional object Syslog-specific decoding options. lossy optional boolean Determines whether or not to replace invalid UTF-8 sequences instead of failing. When true, invalid UTF-8 sequences are replaced with the [`U+FFFD REPLACEMENT CHARACTER`](https://en.wikipedia.org/wiki/Specials_\(Unicode_block\)#Replacement_character). default: truecodec required syslog Decodes the raw bytes as a Syslog message. Decodes either as the [RFC 3164](https://www.ietf.org/rfc/rfc3164.txt)-style format ("old" style) or the [RFC 5424](https://www.ietf.org/rfc/rfc5424.txt)-style format ("new" style, includes structured data). Native optional object Decodes the raw bytes as Vector's [native Protocol Buffers format](https://github.com/vectordotdev/vector/blob/master/lib/vector-core/proto/event.proto). This codec is **[experimental](https://vector.dev/highlights/2022-03-31-native-event-codecs)**. codec required native Decodes the raw bytes as Vector's [native Protocol Buffers format](https://github.com/vectordotdev/vector/blob/master/lib/vector-core/proto/event.proto). This codec is **[experimental](https://vector.dev/highlights/2022-03-31-native-event-codecs)**. NativeJson optional Decodes the raw bytes as Vector's [native JSON format](https://github.com/vectordotdev/vector/blob/master/lib/codecs/tests/data/native_encoding/schema.cue). This codec is **[experimental](https://vector.dev/highlights/2022-03-31-native-event-codecs)**. native_json optional object Vector's native JSON-specific decoding options. lossy optional boolean Determines whether or not to replace invalid UTF-8 sequences instead of failing. When true, invalid UTF-8 sequences are replaced with the [`U+FFFD REPLACEMENT CHARACTER`](https://en.wikipedia.org/wiki/Specials_\(Unicode_block\)#Replacement_character). default: truecodec required native_json Decodes the raw bytes as Vector's [native JSON format](https://github.com/vectordotdev/vector/blob/master/lib/codecs/tests/data/native_encoding/schema.cue). This codec is **[experimental](https://vector.dev/highlights/2022-03-31-native-event-codecs)**. Gelf optional Decodes the raw bytes as a [GELF](https://docs.graylog.org/docs/gelf) message. gelf optional object GELF-specific decoding options. lossy optional boolean Determines whether or not to replace invalid UTF-8 sequences instead of failing. When true, invalid UTF-8 sequences are replaced with the [`U+FFFD REPLACEMENT CHARACTER`](https://en.wikipedia.org/wiki/Specials_\(Unicode_block\)#Replacement_character). default: truecodec required gelf Decodes the raw bytes as a [GELF](https://docs.graylog.org/docs/gelf) message. endpoint optional string The endpoint from which to pull data. framing optional Framing configuration. Framing handles how events are separated when encoded in a raw byte form, where each event is a frame that must be prefixed, or delimited, in a way that marks where an event begins and ends within the byte stream. Bytes optional object Byte frames are passed through as-is according to the underlying I/O boundaries (for example, split between messages or stream segments). method required bytes Byte frames are passed through as-is according to the underlying I/O boundaries (for example, split between messages or stream segments). CharacterDelimited optional Byte frames which are delimited by a chosen character. character_delimited required object Options for the character delimited decoder. delimiter required integer The character that delimits byte sequences. max_length optional integer,​null The maximum length of the byte buffer. This length does *not* include the trailing delimiter. By default, there is no maximum length enforced. If events are malformed, this can lead to additional resource usage as events continue to be buffered in memory, and can potentially lead to memory exhaustion in extreme cases. If there is a risk of processing malformed data, such as logs with user-controlled input, consider setting the maximum length to a reasonably large value as a safety net. This ensures that processing is not actually unbounded. method required character_delimited Byte frames which are delimited by a chosen character. LengthDelimited optional object Byte frames which are prefixed by an unsigned big-endian 32-bit integer indicating the length. method required length_delimited Byte frames which are prefixed by an unsigned big-endian 32-bit integer indicating the length. NewlineDelimited optional Byte frames which are delimited by a newline character. newline_delimited optional object Options for the newline delimited decoder. max_length optional integer,​null The maximum length of the byte buffer. This length does *not* include the trailing delimiter. By default, there is no maximum length enforced. If events are malformed, this can lead to additional resource usage as events continue to be buffered in memory, and can potentially lead to memory exhaustion in extreme cases. If there is a risk of processing malformed data, such as logs with user-controlled input, consider setting the maximum length to a reasonably large value as a safety net. This ensures that processing is not actually unbounded. method required newline_delimited Byte frames which are delimited by a newline character. OctetCounting optional Byte frames according to the [octet counting](https://tools.ietf.org/html/rfc6587#section-3.4.1) format. octet_counting optional object Options for the octet counting decoder. max_length optional integer,​null The maximum length of the byte buffer. method required octet_counting Byte frames according to the [octet counting](https://tools.ietf.org/html/rfc6587#section-3.4.1) format. full_response_size optional integer The number of messages in a response to mark a stream as "busy". This is used to determine if more streams should be started. The GCP Pub/Sub servers send responses with 100 or more messages when the subscription is busy. keepalive_secs optional number The amount of time, in seconds, with no received activity before sending a keepalive request. If this is set larger than `60`, you may see periodic errors sent from the server. log_namespace optional boolean,​null The namespace to use for logs. This overrides the global setting. max_concurrency optional integer The maximum number of concurrent stream connections to open at once. poll_time_seconds optional number How often to poll the currently active streams to see if they are all busy and so open a new stream. project required string The project name from which to pull logs. retry_delay_seconds optional number,​null **DEPRECATED**: The amount of time, in seconds, to wait between retry attempts after an error. retry_delay_secs optional number The amount of time, in seconds, to wait between retry attempts after an error. subscription required string The subscription within the project which is configured to receive logs. tls optional TLS configuration. Option 1 optional object TLS configuration. alpn_protocols optional array,​null Sets the list of supported ALPN protocols. Declare the supported ALPN protocols, which are used during negotiation with peer. They are prioritized in the order that they are defined. ca_file optional Absolute path to an additional CA certificate file. The certificate must be in the DER or PEM (X.509) format. Additionally, the certificate can be provided as an inline string in PEM format. Option 1 optional string A file path. crt_file optional Absolute path to a certificate file used to identify this server. The certificate must be in DER, PEM (X.509), or PKCS#12 format. Additionally, the certificate can be provided as an inline string in PEM format. If this is set, and is not a PKCS#12 archive, `key_file` must also be set. Option 1 optional string A file path. key_file optional Absolute path to a private key file used to identify this server. The key must be in DER or PEM (PKCS#8) format. Additionally, the key can be provided as an inline string in PEM format. Option 1 optional string A file path. key_pass optional string,​null Passphrase used to unlock the encrypted key file. This has no effect unless `key_file` is set. verify_certificate optional boolean,​null Enables certificate verification. If enabled, certificates must not be expired and must be issued by a trusted issuer. This verification operates in a hierarchical manner, checking that the leaf certificate (the certificate presented by the client/server) is not only valid, but that the issuer of that certificate is also valid, and so on until the verification process reaches a root certificate. Relevant for both incoming and outgoing connections. Do NOT set this to `false` unless you understand the risks of not verifying the validity of certificates. verify_hostname optional boolean,​null Enables hostname verification. If enabled, the hostname used to connect to the remote host must be present in the TLS certificate presented by the remote host, either as the Common Name or as an entry in the Subject Alternative Name extension. Only relevant for outgoing connections. Do NOT set this to `false` unless you understand the risks of not verifying the remote hostname. api_key optional An [API key](https://cloud.google.com/docs/authentication/api-keys). Either an API key or a path to a service account credentials JSON file can be specified. If both are unset, the `GOOGLE_APPLICATION_CREDENTIALS` environment variable is checked for a filename. If no filename is named, an attempt is made to fetch an instance service account for the compute instance the program is running on. If this is not on a GCE instance, then you must define it with an API key or service account credentials JSON file. Option 1 optional string Wrapper for sensitive strings containing credentials credentials_path optional string,​null Path to a [service account](https://cloud.google.com/docs/authentication/production#manually) credentials JSON file. Either an API key or a path to a service account credentials JSON file can be specified. If both are unset, the `GOOGLE_APPLICATION_CREDENTIALS` environment variable is checked for a filename. If no filename is named, an attempt is made to fetch an instance service account for the compute instance the program is running on. If this is not on a GCE instance, then you must define it with an API key or service account credentials JSON file. skip_authentication optional boolean Skip all authentication handling. For use with integration tests only. {% /tab %} {% tab title="pubsub-request-example" %} ```yaml ack_deadline_seconds: integer ack_deadline_secs: 600 acknowledgements: enabled: null decoding: codec: bytes endpoint: 'https://pubsub.googleapis.com' framing: method: bytes full_response_size: 100 keepalive_secs: 60 log_namespace: boolean max_concurrency: 10 poll_time_seconds: 2 project: string retry_delay_seconds: number retry_delay_secs: 1 subscription: string tls: '' type: gcp_pubsub ``` {% /tab %} ### Heroku Logs{% #logplex %} Configuration for `heroku_logs` source. OptionsSchema {% tab title="logplex-request-model" %} FieldrequiredTypeDescription acknowledgements optional object Controls how acknowledgements are handled by this source. **DEPRECATED**: This setting is **deprecated** in favor of enabling `acknowledgements` at the [global](https://vector.dev/docs/reference/configuration/global-options/#acknowledgements) or sink level. Enabling or disabling acknowledgements at the source level has **no effect** on acknowledgement behavior. See [End-to-end Acknowledgements](https://vector.dev/docs/about/under-the-hood/architecture/end-to-end-acknowledgements/) for more information on how event acknowledgement is handled. enabled optional boolean,​null Whether or not end-to-end acknowledgements are enabled for this source. default: nulladdress required string The socket address to listen for connections on. auth optional HTTP Basic authentication configuration. Option 1 optional object HTTP Basic authentication configuration. password required string The password for basic authentication. username required string The username for basic authentication. decoding optional Configures how events are decoded from raw bytes. Bytes optional object Uses the raw bytes as-is. codec required bytes Uses the raw bytes as-is. Json optional Decodes the raw bytes as [JSON](https://www.json.org/). json optional object JSON-specific decoding options. lossy optional boolean Determines whether or not to replace invalid UTF-8 sequences instead of failing. When true, invalid UTF-8 sequences are replaced with the [`U+FFFD REPLACEMENT CHARACTER`](https://en.wikipedia.org/wiki/Specials_\(Unicode_block\)#Replacement_character). default: truecodec required json Decodes the raw bytes as [JSON](https://www.json.org/). Protobuf optional Decodes the raw bytes as [protobuf](https://protobuf.dev/). protobuf optional object Protobuf-specific decoding options. desc_file required string Path to desc file message_type required string message type. e.g package.message codec required protobuf Decodes the raw bytes as [protobuf](https://protobuf.dev/). Syslog optional Decodes the raw bytes as a Syslog message. Decodes either as the [RFC 3164](https://www.ietf.org/rfc/rfc3164.txt)-style format ("old" style) or the [RFC 5424](https://www.ietf.org/rfc/rfc5424.txt)-style format ("new" style, includes structured data). syslog optional object Syslog-specific decoding options. lossy optional boolean Determines whether or not to replace invalid UTF-8 sequences instead of failing. When true, invalid UTF-8 sequences are replaced with the [`U+FFFD REPLACEMENT CHARACTER`](https://en.wikipedia.org/wiki/Specials_\(Unicode_block\)#Replacement_character). default: truecodec required syslog Decodes the raw bytes as a Syslog message. Decodes either as the [RFC 3164](https://www.ietf.org/rfc/rfc3164.txt)-style format ("old" style) or the [RFC 5424](https://www.ietf.org/rfc/rfc5424.txt)-style format ("new" style, includes structured data). Native optional object Decodes the raw bytes as Vector's [native Protocol Buffers format](https://github.com/vectordotdev/vector/blob/master/lib/vector-core/proto/event.proto). This codec is **[experimental](https://vector.dev/highlights/2022-03-31-native-event-codecs)**. codec required native Decodes the raw bytes as Vector's [native Protocol Buffers format](https://github.com/vectordotdev/vector/blob/master/lib/vector-core/proto/event.proto). This codec is **[experimental](https://vector.dev/highlights/2022-03-31-native-event-codecs)**. NativeJson optional Decodes the raw bytes as Vector's [native JSON format](https://github.com/vectordotdev/vector/blob/master/lib/codecs/tests/data/native_encoding/schema.cue). This codec is **[experimental](https://vector.dev/highlights/2022-03-31-native-event-codecs)**. native_json optional object Vector's native JSON-specific decoding options. lossy optional boolean Determines whether or not to replace invalid UTF-8 sequences instead of failing. When true, invalid UTF-8 sequences are replaced with the [`U+FFFD REPLACEMENT CHARACTER`](https://en.wikipedia.org/wiki/Specials_\(Unicode_block\)#Replacement_character). default: truecodec required native_json Decodes the raw bytes as Vector's [native JSON format](https://github.com/vectordotdev/vector/blob/master/lib/codecs/tests/data/native_encoding/schema.cue). This codec is **[experimental](https://vector.dev/highlights/2022-03-31-native-event-codecs)**. Gelf optional Decodes the raw bytes as a [GELF](https://docs.graylog.org/docs/gelf) message. gelf optional object GELF-specific decoding options. lossy optional boolean Determines whether or not to replace invalid UTF-8 sequences instead of failing. When true, invalid UTF-8 sequences are replaced with the [`U+FFFD REPLACEMENT CHARACTER`](https://en.wikipedia.org/wiki/Specials_\(Unicode_block\)#Replacement_character). default: truecodec required gelf Decodes the raw bytes as a [GELF](https://docs.graylog.org/docs/gelf) message. framing optional Framing configuration. Framing handles how events are separated when encoded in a raw byte form, where each event is a frame that must be prefixed, or delimited, in a way that marks where an event begins and ends within the byte stream. Bytes optional object Byte frames are passed through as-is according to the underlying I/O boundaries (for example, split between messages or stream segments). method required bytes Byte frames are passed through as-is according to the underlying I/O boundaries (for example, split between messages or stream segments). CharacterDelimited optional Byte frames which are delimited by a chosen character. character_delimited required object Options for the character delimited decoder. delimiter required integer The character that delimits byte sequences. max_length optional integer,​null The maximum length of the byte buffer. This length does *not* include the trailing delimiter. By default, there is no maximum length enforced. If events are malformed, this can lead to additional resource usage as events continue to be buffered in memory, and can potentially lead to memory exhaustion in extreme cases. If there is a risk of processing malformed data, such as logs with user-controlled input, consider setting the maximum length to a reasonably large value as a safety net. This ensures that processing is not actually unbounded. method required character_delimited Byte frames which are delimited by a chosen character. LengthDelimited optional object Byte frames which are prefixed by an unsigned big-endian 32-bit integer indicating the length. method required length_delimited Byte frames which are prefixed by an unsigned big-endian 32-bit integer indicating the length. NewlineDelimited optional Byte frames which are delimited by a newline character. newline_delimited optional object Options for the newline delimited decoder. max_length optional integer,​null The maximum length of the byte buffer. This length does *not* include the trailing delimiter. By default, there is no maximum length enforced. If events are malformed, this can lead to additional resource usage as events continue to be buffered in memory, and can potentially lead to memory exhaustion in extreme cases. If there is a risk of processing malformed data, such as logs with user-controlled input, consider setting the maximum length to a reasonably large value as a safety net. This ensures that processing is not actually unbounded. method required newline_delimited Byte frames which are delimited by a newline character. OctetCounting optional Byte frames according to the [octet counting](https://tools.ietf.org/html/rfc6587#section-3.4.1) format. octet_counting optional object Options for the octet counting decoder. max_length optional integer,​null The maximum length of the byte buffer. method required octet_counting Byte frames according to the [octet counting](https://tools.ietf.org/html/rfc6587#section-3.4.1) format. log_namespace optional boolean,​null The namespace to use for logs. This overrides the global setting. query_parameters optional [string] A list of URL query parameters to include in the log event. These override any values included in the body with conflicting names. tls optional Configures the TLS options for incoming/outgoing connections. Option 1 optional Configures the TLS options for incoming/outgoing connections. enabled optional boolean,​null Whether or not to require TLS for incoming or outgoing connections. When enabled and used for incoming connections, an identity certificate is also required. See `tls.crt_file` for more information. alpn_protocols optional array,​null Sets the list of supported ALPN protocols. Declare the supported ALPN protocols, which are used during negotiation with peer. They are prioritized in the order that they are defined. ca_file optional Absolute path to an additional CA certificate file. The certificate must be in the DER or PEM (X.509) format. Additionally, the certificate can be provided as an inline string in PEM format. Option 1 optional string A file path. crt_file optional Absolute path to a certificate file used to identify this server. The certificate must be in DER, PEM (X.509), or PKCS#12 format. Additionally, the certificate can be provided as an inline string in PEM format. If this is set, and is not a PKCS#12 archive, `key_file` must also be set. Option 1 optional string A file path. key_file optional Absolute path to a private key file used to identify this server. The key must be in DER or PEM (PKCS#8) format. Additionally, the key can be provided as an inline string in PEM format. Option 1 optional string A file path. key_pass optional string,​null Passphrase used to unlock the encrypted key file. This has no effect unless `key_file` is set. verify_certificate optional boolean,​null Enables certificate verification. If enabled, certificates must not be expired and must be issued by a trusted issuer. This verification operates in a hierarchical manner, checking that the leaf certificate (the certificate presented by the client/server) is not only valid, but that the issuer of that certificate is also valid, and so on until the verification process reaches a root certificate. Relevant for both incoming and outgoing connections. Do NOT set this to `false` unless you understand the risks of not verifying the validity of certificates. verify_hostname optional boolean,​null Enables hostname verification. If enabled, the hostname used to connect to the remote host must be present in the TLS certificate presented by the remote host, either as the Common Name or as an entry in the Subject Alternative Name extension. Only relevant for outgoing connections. Do NOT set this to `false` unless you understand the risks of not verifying the remote hostname. {% /tab %} {% tab title="logplex-request-example" %} ```yaml acknowledgements: enabled: null address: string auth: '' decoding: codec: bytes framing: method: bytes log_namespace: boolean query_parameters: [] tls: '' type: heroku_logs ``` {% /tab %} ### Host Metrics{% #hostmetrics %} Configuration for the `host_metrics` source. OptionsSchema {% tab title="hostmetrics-request-model" %} FieldrequiredTypeDescription cgroups optional Options for the cgroups (controller groups) metrics collector. This collector is only available on Linux systems, and only supports either version 2 or hybrid cgroups. Option 1 optional object Options for the cgroups (controller groups) metrics collector. This collector is only available on Linux systems, and only supports either version 2 or hybrid cgroups. base optional The base cgroup name to provide metrics for. default: nullOption 1 optional string A file path. base_dir optional Base cgroup directory, for testing use only Option 1 optional string A file path. groups optional object Lists of cgroup name patterns to include or exclude in gathering usage metrics. default: {"excludes":null,"includes":null}excludes optional array,​null Any patterns which should be excluded. The patterns are matched using globbing. default: nullincludes optional array,​null Any patterns which should be included. The patterns are matched using globbing. default: ["*"]levels optional integer The number of levels of the cgroups hierarchy for which to report metrics. A value of `1` means the root or named cgroup. default: 100collectors optional array,​null The list of host metric collector services to use. Defaults to all collectors. disk optional object Options for the disk metrics collector. devices optional object Lists of device name patterns to include or exclude in gathering I/O utilization metrics. default: {"excludes":null,"includes":null}excludes optional array,​null Any patterns which should be excluded. The patterns are matched using globbing. default: nullincludes optional array,​null Any patterns which should be included. The patterns are matched using globbing. default: ["*"] filesystem optional object Options for the filesystem metrics collector. devices optional object Lists of device name patterns to include or exclude in gathering usage metrics. default: {"excludes":null,"includes":null}excludes optional array,​null Any patterns which should be excluded. The patterns are matched using globbing. default: nullincludes optional array,​null Any patterns which should be included. The patterns are matched using globbing. default: ["*"] filesystems optional object Lists of filesystem name patterns to include or exclude in gathering usage metrics. default: {"excludes":null,"includes":null}excludes optional array,​null Any patterns which should be excluded. The patterns are matched using globbing. default: nullincludes optional array,​null Any patterns which should be included. The patterns are matched using globbing. default: ["*"] mountpoints optional object Lists of mount point path patterns to include or exclude in gathering usage metrics. default: {"excludes":null,"includes":null}excludes optional array,​null Any patterns which should be excluded. The patterns are matched using globbing. default: nullincludes optional array,​null Any patterns which should be included. The patterns are matched using globbing. default: ["*"]namespace optional string,​null Overrides the default namespace for the metrics emitted by the source. network optional object Options for the network metrics collector. devices optional object Lists of device name patterns to include or exclude in gathering network utilization metrics. default: {"excludes":null,"includes":null}excludes optional array,​null Any patterns which should be excluded. The patterns are matched using globbing. default: nullincludes optional array,​null Any patterns which should be included. The patterns are matched using globbing. default: ["*"]scrape_interval_secs optional integer The interval between metric gathering, in seconds. {% /tab %} {% tab title="hostmetrics-request-example" %} ```yaml cgroups: base: null groups: excludes: null includes: null levels: 100 collectors: - cpu - disk - filesystem - load - host - memory - network - cgroups disk: devices: excludes: null includes: null filesystem: devices: excludes: null includes: null filesystems: excludes: null includes: null mountpoints: excludes: null includes: null namespace: host network: devices: excludes: null includes: null scrape_interval_secs: 15 type: host_metrics ``` {% /tab %} ### HTTP{% #http %} Configuration for the `http` source. OptionsSchema {% tab title="http-request-model" %} FieldrequiredTypeDescription acknowledgements optional object Controls how acknowledgements are handled by this source. **DEPRECATED**: This setting is **deprecated** in favor of enabling `acknowledgements` at the [global](https://vector.dev/docs/reference/configuration/global-options/#acknowledgements) or sink level. Enabling or disabling acknowledgements at the source level has **no effect** on acknowledgement behavior. See [End-to-end Acknowledgements](https://vector.dev/docs/about/under-the-hood/architecture/end-to-end-acknowledgements/) for more information on how event acknowledgement is handled. enabled optional boolean,​null Whether or not end-to-end acknowledgements are enabled for this source. default: nulladdress required string The socket address to listen for connections on. It *must* include a port. auth optional HTTP Basic authentication configuration. Option 1 optional object HTTP Basic authentication configuration. password required string The password for basic authentication. username required string The username for basic authentication. decoding optional Configures how events are decoded from raw bytes. Option 1 optional Configures how events are decoded from raw bytes. Bytes optional object Uses the raw bytes as-is. codec required bytes Uses the raw bytes as-is. Json optional Decodes the raw bytes as [JSON](https://www.json.org/). json optional object JSON-specific decoding options. lossy optional boolean Determines whether or not to replace invalid UTF-8 sequences instead of failing. When true, invalid UTF-8 sequences are replaced with the [`U+FFFD REPLACEMENT CHARACTER`](https://en.wikipedia.org/wiki/Specials_\(Unicode_block\)#Replacement_character). default: truecodec required json Decodes the raw bytes as [JSON](https://www.json.org/). Protobuf optional Decodes the raw bytes as [protobuf](https://protobuf.dev/). protobuf optional object Protobuf-specific decoding options. desc_file required string Path to desc file message_type required string message type. e.g package.message codec required protobuf Decodes the raw bytes as [protobuf](https://protobuf.dev/). Syslog optional Decodes the raw bytes as a Syslog message. Decodes either as the [RFC 3164](https://www.ietf.org/rfc/rfc3164.txt)-style format ("old" style) or the [RFC 5424](https://www.ietf.org/rfc/rfc5424.txt)-style format ("new" style, includes structured data). syslog optional object Syslog-specific decoding options. lossy optional boolean Determines whether or not to replace invalid UTF-8 sequences instead of failing. When true, invalid UTF-8 sequences are replaced with the [`U+FFFD REPLACEMENT CHARACTER`](https://en.wikipedia.org/wiki/Specials_\(Unicode_block\)#Replacement_character). default: truecodec required syslog Decodes the raw bytes as a Syslog message. Decodes either as the [RFC 3164](https://www.ietf.org/rfc/rfc3164.txt)-style format ("old" style) or the [RFC 5424](https://www.ietf.org/rfc/rfc5424.txt)-style format ("new" style, includes structured data). Native optional object Decodes the raw bytes as Vector's [native Protocol Buffers format](https://github.com/vectordotdev/vector/blob/master/lib/vector-core/proto/event.proto). This codec is **[experimental](https://vector.dev/highlights/2022-03-31-native-event-codecs)**. codec required native Decodes the raw bytes as Vector's [native Protocol Buffers format](https://github.com/vectordotdev/vector/blob/master/lib/vector-core/proto/event.proto). This codec is **[experimental](https://vector.dev/highlights/2022-03-31-native-event-codecs)**. NativeJson optional Decodes the raw bytes as Vector's [native JSON format](https://github.com/vectordotdev/vector/blob/master/lib/codecs/tests/data/native_encoding/schema.cue). This codec is **[experimental](https://vector.dev/highlights/2022-03-31-native-event-codecs)**. native_json optional object Vector's native JSON-specific decoding options. lossy optional boolean Determines whether or not to replace invalid UTF-8 sequences instead of failing. When true, invalid UTF-8 sequences are replaced with the [`U+FFFD REPLACEMENT CHARACTER`](https://en.wikipedia.org/wiki/Specials_\(Unicode_block\)#Replacement_character). default: truecodec required native_json Decodes the raw bytes as Vector's [native JSON format](https://github.com/vectordotdev/vector/blob/master/lib/codecs/tests/data/native_encoding/schema.cue). This codec is **[experimental](https://vector.dev/highlights/2022-03-31-native-event-codecs)**. Gelf optional Decodes the raw bytes as a [GELF](https://docs.graylog.org/docs/gelf) message. gelf optional object GELF-specific decoding options. lossy optional boolean Determines whether or not to replace invalid UTF-8 sequences instead of failing. When true, invalid UTF-8 sequences are replaced with the [`U+FFFD REPLACEMENT CHARACTER`](https://en.wikipedia.org/wiki/Specials_\(Unicode_block\)#Replacement_character). default: truecodec required gelf Decodes the raw bytes as a [GELF](https://docs.graylog.org/docs/gelf) message. encoding optional The expected encoding of received data. For `json` and `ndjson` encodings, the fields of the JSON objects are output as separate fields. Option 1 optional Content encoding. text optional text Plaintext. ndjson optional ndjson Newline-delimited JSON. json optional json JSON. binary optional binary Binary. framing optional Framing configuration. Framing handles how events are separated when encoded in a raw byte form, where each event is a frame that must be prefixed, or delimited, in a way that marks where an event begins and ends within the byte stream. Option 1 optional Framing configuration. Framing handles how events are separated when encoded in a raw byte form, where each event is a frame that must be prefixed, or delimited, in a way that marks where an event begins and ends within the byte stream. Bytes optional object Byte frames are passed through as-is according to the underlying I/O boundaries (for example, split between messages or stream segments). method required bytes Byte frames are passed through as-is according to the underlying I/O boundaries (for example, split between messages or stream segments). CharacterDelimited optional Byte frames which are delimited by a chosen character. character_delimited required object Options for the character delimited decoder. delimiter required integer The character that delimits byte sequences. max_length optional integer,​null The maximum length of the byte buffer. This length does *not* include the trailing delimiter. By default, there is no maximum length enforced. If events are malformed, this can lead to additional resource usage as events continue to be buffered in memory, and can potentially lead to memory exhaustion in extreme cases. If there is a risk of processing malformed data, such as logs with user-controlled input, consider setting the maximum length to a reasonably large value as a safety net. This ensures that processing is not actually unbounded. method required character_delimited Byte frames which are delimited by a chosen character. LengthDelimited optional object Byte frames which are prefixed by an unsigned big-endian 32-bit integer indicating the length. method required length_delimited Byte frames which are prefixed by an unsigned big-endian 32-bit integer indicating the length. NewlineDelimited optional Byte frames which are delimited by a newline character. newline_delimited optional object Options for the newline delimited decoder. max_length optional integer,​null The maximum length of the byte buffer. This length does *not* include the trailing delimiter. By default, there is no maximum length enforced. If events are malformed, this can lead to additional resource usage as events continue to be buffered in memory, and can potentially lead to memory exhaustion in extreme cases. If there is a risk of processing malformed data, such as logs with user-controlled input, consider setting the maximum length to a reasonably large value as a safety net. This ensures that processing is not actually unbounded. method required newline_delimited Byte frames which are delimited by a newline character. OctetCounting optional Byte frames according to the [octet counting](https://tools.ietf.org/html/rfc6587#section-3.4.1) format. octet_counting optional object Options for the octet counting decoder. max_length optional integer,​null The maximum length of the byte buffer. method required octet_counting Byte frames according to the [octet counting](https://tools.ietf.org/html/rfc6587#section-3.4.1) format. headers optional [string] A list of HTTP headers to include in the log event. These override any values included in the JSON payload with conflicting names. log_namespace optional boolean,​null The namespace to use for logs. This overrides the global setting. method optional Specifies the action of the HTTP request. HEAD optional HEAD HTTP HEAD method. GET optional GET HTTP GET method. POST optional POST HTTP POST method. PUT optional PUT HTTP Put method. PATCH optional PATCH HTTP PATCH method. DELETE optional DELETE HTTP DELETE method. path optional string The URL path on which log event POST requests are sent. path_key optional string The event key in which the requested URL path used to send the request is stored. query_parameters optional [string] A list of URL query parameters to include in the log event. These override any values included in the body with conflicting names. response_code optional integer Specifies the HTTP response status code that will be returned on successful requests. strict_path optional boolean Whether or not to treat the configured `path` as an absolute path. If set to `true`, only requests using the exact URL path specified in `path` are accepted. Otherwise, requests sent to a URL path that starts with the value of `path` are accepted. With `strict_path` set to `false` and `path` set to `""`, the configured HTTP source accepts requests from any URL path. tls optional Configures the TLS options for incoming/outgoing connections. Option 1 optional Configures the TLS options for incoming/outgoing connections. enabled optional boolean,​null Whether or not to require TLS for incoming or outgoing connections. When enabled and used for incoming connections, an identity certificate is also required. See `tls.crt_file` for more information. alpn_protocols optional array,​null Sets the list of supported ALPN protocols. Declare the supported ALPN protocols, which are used during negotiation with peer. They are prioritized in the order that they are defined. ca_file optional Absolute path to an additional CA certificate file. The certificate must be in the DER or PEM (X.509) format. Additionally, the certificate can be provided as an inline string in PEM format. Option 1 optional string A file path. crt_file optional Absolute path to a certificate file used to identify this server. The certificate must be in DER, PEM (X.509), or PKCS#12 format. Additionally, the certificate can be provided as an inline string in PEM format. If this is set, and is not a PKCS#12 archive, `key_file` must also be set. Option 1 optional string A file path. key_file optional Absolute path to a private key file used to identify this server. The key must be in DER or PEM (PKCS#8) format. Additionally, the key can be provided as an inline string in PEM format. Option 1 optional string A file path. key_pass optional string,​null Passphrase used to unlock the encrypted key file. This has no effect unless `key_file` is set. verify_certificate optional boolean,​null Enables certificate verification. If enabled, certificates must not be expired and must be issued by a trusted issuer. This verification operates in a hierarchical manner, checking that the leaf certificate (the certificate presented by the client/server) is not only valid, but that the issuer of that certificate is also valid, and so on until the verification process reaches a root certificate. Relevant for both incoming and outgoing connections. Do NOT set this to `false` unless you understand the risks of not verifying the validity of certificates. verify_hostname optional boolean,​null Enables hostname verification. If enabled, the hostname used to connect to the remote host must be present in the TLS certificate presented by the remote host, either as the Common Name or as an entry in the Subject Alternative Name extension. Only relevant for outgoing connections. Do NOT set this to `false` unless you understand the risks of not verifying the remote hostname. {% /tab %} {% tab title="http-request-example" %} ```yaml acknowledgements: enabled: null address: string auth: '' decoding: '' encoding: '' framing: '' headers: [] log_namespace: boolean method: POST path: / path_key: path query_parameters: [] response_code: 200 strict_path: true tls: '' type: http ``` {% /tab %} ### HTTP Client{% #httpclient %} Configuration for the `http_client` source. OptionsSchema {% tab title="httpclient-request-model" %} FieldrequiredTypeDescription auth optional Configuration of the authentication strategy for HTTP requests. HTTP Authentication. Option 1 optional Configuration of the authentication strategy for HTTP requests. HTTP authentication should be used with HTTPS only, as the authentication credentials are passed as an HTTP header without any additional encryption beyond what is provided by the transport itself. Basic optional object Basic authentication. The username and password are concatenated and encoded via [base64](https://en.wikipedia.org/wiki/Base64). password required string The basic authentication password. strategy required basic Basic authentication. The username and password are concatenated and encoded via [base64](https://en.wikipedia.org/wiki/Base64). user required string The basic authentication username. Bearer optional object Bearer authentication. The bearer token value (OAuth2, JWT, etc.) is passed as-is. strategy required bearer Bearer authentication. The bearer token value (OAuth2, JWT, etc.) is passed as-is. token required string The bearer authentication token. decoding optional Decoder to use on the HTTP responses. Bytes optional object Uses the raw bytes as-is. codec required bytes Uses the raw bytes as-is. Json optional Decodes the raw bytes as [JSON](https://www.json.org/). json optional object JSON-specific decoding options. lossy optional boolean Determines whether or not to replace invalid UTF-8 sequences instead of failing. When true, invalid UTF-8 sequences are replaced with the [`U+FFFD REPLACEMENT CHARACTER`](https://en.wikipedia.org/wiki/Specials_\(Unicode_block\)#Replacement_character). default: truecodec required json Decodes the raw bytes as [JSON](https://www.json.org/). Protobuf optional Decodes the raw bytes as [protobuf](https://protobuf.dev/). protobuf optional object Protobuf-specific decoding options. desc_file required string Path to desc file message_type required string message type. e.g package.message codec required protobuf Decodes the raw bytes as [protobuf](https://protobuf.dev/). Syslog optional Decodes the raw bytes as a Syslog message. Decodes either as the [RFC 3164](https://www.ietf.org/rfc/rfc3164.txt)-style format ("old" style) or the [RFC 5424](https://www.ietf.org/rfc/rfc5424.txt)-style format ("new" style, includes structured data). syslog optional object Syslog-specific decoding options. lossy optional boolean Determines whether or not to replace invalid UTF-8 sequences instead of failing. When true, invalid UTF-8 sequences are replaced with the [`U+FFFD REPLACEMENT CHARACTER`](https://en.wikipedia.org/wiki/Specials_\(Unicode_block\)#Replacement_character). default: truecodec required syslog Decodes the raw bytes as a Syslog message. Decodes either as the [RFC 3164](https://www.ietf.org/rfc/rfc3164.txt)-style format ("old" style) or the [RFC 5424](https://www.ietf.org/rfc/rfc5424.txt)-style format ("new" style, includes structured data). Native optional object Decodes the raw bytes as Vector's [native Protocol Buffers format](https://github.com/vectordotdev/vector/blob/master/lib/vector-core/proto/event.proto). This codec is **[experimental](https://vector.dev/highlights/2022-03-31-native-event-codecs)**. codec required native Decodes the raw bytes as Vector's [native Protocol Buffers format](https://github.com/vectordotdev/vector/blob/master/lib/vector-core/proto/event.proto). This codec is **[experimental](https://vector.dev/highlights/2022-03-31-native-event-codecs)**. NativeJson optional Decodes the raw bytes as Vector's [native JSON format](https://github.com/vectordotdev/vector/blob/master/lib/codecs/tests/data/native_encoding/schema.cue). This codec is **[experimental](https://vector.dev/highlights/2022-03-31-native-event-codecs)**. native_json optional object Vector's native JSON-specific decoding options. lossy optional boolean Determines whether or not to replace invalid UTF-8 sequences instead of failing. When true, invalid UTF-8 sequences are replaced with the [`U+FFFD REPLACEMENT CHARACTER`](https://en.wikipedia.org/wiki/Specials_\(Unicode_block\)#Replacement_character). default: truecodec required native_json Decodes the raw bytes as Vector's [native JSON format](https://github.com/vectordotdev/vector/blob/master/lib/codecs/tests/data/native_encoding/schema.cue). This codec is **[experimental](https://vector.dev/highlights/2022-03-31-native-event-codecs)**. Gelf optional Decodes the raw bytes as a [GELF](https://docs.graylog.org/docs/gelf) message. gelf optional object GELF-specific decoding options. lossy optional boolean Determines whether or not to replace invalid UTF-8 sequences instead of failing. When true, invalid UTF-8 sequences are replaced with the [`U+FFFD REPLACEMENT CHARACTER`](https://en.wikipedia.org/wiki/Specials_\(Unicode_block\)#Replacement_character). default: truecodec required gelf Decodes the raw bytes as a [GELF](https://docs.graylog.org/docs/gelf) message. endpoint required string The HTTP endpoint to collect events from. The full path must be specified. framing optional Framing configuration. Framing to use in the decoding. Bytes optional object Byte frames are passed through as-is according to the underlying I/O boundaries (for example, split between messages or stream segments). method required bytes Byte frames are passed through as-is according to the underlying I/O boundaries (for example, split between messages or stream segments). CharacterDelimited optional Byte frames which are delimited by a chosen character. character_delimited required object Options for the character delimited decoder. delimiter required integer The character that delimits byte sequences. max_length optional integer,​null The maximum length of the byte buffer. This length does *not* include the trailing delimiter. By default, there is no maximum length enforced. If events are malformed, this can lead to additional resource usage as events continue to be buffered in memory, and can potentially lead to memory exhaustion in extreme cases. If there is a risk of processing malformed data, such as logs with user-controlled input, consider setting the maximum length to a reasonably large value as a safety net. This ensures that processing is not actually unbounded. method required character_delimited Byte frames which are delimited by a chosen character. LengthDelimited optional object Byte frames which are prefixed by an unsigned big-endian 32-bit integer indicating the length. method required length_delimited Byte frames which are prefixed by an unsigned big-endian 32-bit integer indicating the length. NewlineDelimited optional Byte frames which are delimited by a newline character. newline_delimited optional object Options for the newline delimited decoder. max_length optional integer,​null The maximum length of the byte buffer. This length does *not* include the trailing delimiter. By default, there is no maximum length enforced. If events are malformed, this can lead to additional resource usage as events continue to be buffered in memory, and can potentially lead to memory exhaustion in extreme cases. If there is a risk of processing malformed data, such as logs with user-controlled input, consider setting the maximum length to a reasonably large value as a safety net. This ensures that processing is not actually unbounded. method required newline_delimited Byte frames which are delimited by a newline character. OctetCounting optional Byte frames according to the [octet counting](https://tools.ietf.org/html/rfc6587#section-3.4.1) format. octet_counting optional object Options for the octet counting decoder. max_length optional integer,​null The maximum length of the byte buffer. method required octet_counting Byte frames according to the [octet counting](https://tools.ietf.org/html/rfc6587#section-3.4.1) format. headers optional object Headers to apply to the HTTP requests. One or more values for the same header can be provided. log_namespace optional boolean,​null The namespace to use for logs. This overrides the global setting. method optional Specifies the method of the HTTP request. HEAD optional HEAD HTTP HEAD method. GET optional GET HTTP GET method. POST optional POST HTTP POST method. PUT optional PUT HTTP Put method. PATCH optional PATCH HTTP PATCH method. DELETE optional DELETE HTTP DELETE method. query optional object Custom parameters for the HTTP request query string. One or more values for the same parameter key can be provided. The parameters provided in this option are appended to any parameters manually provided in the `endpoint` option. scrape_interval_secs optional integer The interval between scrapes. Requests are run concurrently so if a scrape takes longer than the interval a new scrape will be started. This can take extra resources, set the timeout to a value lower than the scrape interval to prevent this from happening. scrape_timeout_secs optional number The timeout for each scrape request. tls optional TLS configuration. Option 1 optional object TLS configuration. alpn_protocols optional array,​null Sets the list of supported ALPN protocols. Declare the supported ALPN protocols, which are used during negotiation with peer. They are prioritized in the order that they are defined. ca_file optional Absolute path to an additional CA certificate file. The certificate must be in the DER or PEM (X.509) format. Additionally, the certificate can be provided as an inline string in PEM format. Option 1 optional string A file path. crt_file optional Absolute path to a certificate file used to identify this server. The certificate must be in DER, PEM (X.509), or PKCS#12 format. Additionally, the certificate can be provided as an inline string in PEM format. If this is set, and is not a PKCS#12 archive, `key_file` must also be set. Option 1 optional string A file path. key_file optional Absolute path to a private key file used to identify this server. The key must be in DER or PEM (PKCS#8) format. Additionally, the key can be provided as an inline string in PEM format. Option 1 optional string A file path. key_pass optional string,​null Passphrase used to unlock the encrypted key file. This has no effect unless `key_file` is set. verify_certificate optional boolean,​null Enables certificate verification. If enabled, certificates must not be expired and must be issued by a trusted issuer. This verification operates in a hierarchical manner, checking that the leaf certificate (the certificate presented by the client/server) is not only valid, but that the issuer of that certificate is also valid, and so on until the verification process reaches a root certificate. Relevant for both incoming and outgoing connections. Do NOT set this to `false` unless you understand the risks of not verifying the validity of certificates. verify_hostname optional boolean,​null Enables hostname verification. If enabled, the hostname used to connect to the remote host must be present in the TLS certificate presented by the remote host, either as the Common Name or as an entry in the Subject Alternative Name extension. Only relevant for outgoing connections. Do NOT set this to `false` unless you understand the risks of not verifying the remote hostname. {% /tab %} {% tab title="httpclient-request-example" %} ```yaml auth: '' decoding: codec: bytes endpoint: string framing: method: bytes headers: {} log_namespace: boolean method: GET query: {} scrape_interval_secs: 15 scrape_timeout_secs: 5 tls: '' type: http_client ``` {% /tab %} ### HTTP Server{% #simplehttp %} Configuration for the `http_server` source. OptionsSchema {% tab title="simplehttp-request-model" %} FieldrequiredTypeDescription acknowledgements optional object Controls how acknowledgements are handled by this source. **DEPRECATED**: This setting is **deprecated** in favor of enabling `acknowledgements` at the [global](https://vector.dev/docs/reference/configuration/global-options/#acknowledgements) or sink level. Enabling or disabling acknowledgements at the source level has **no effect** on acknowledgement behavior. See [End-to-end Acknowledgements](https://vector.dev/docs/about/under-the-hood/architecture/end-to-end-acknowledgements/) for more information on how event acknowledgement is handled. enabled optional boolean,​null Whether or not end-to-end acknowledgements are enabled for this source. default: nulladdress required string The socket address to listen for connections on. It *must* include a port. auth optional HTTP Basic authentication configuration. Option 1 optional object HTTP Basic authentication configuration. password required string The password for basic authentication. username required string The username for basic authentication. decoding optional Configures how events are decoded from raw bytes. Option 1 optional Configures how events are decoded from raw bytes. Bytes optional object Uses the raw bytes as-is. codec required bytes Uses the raw bytes as-is. Json optional Decodes the raw bytes as [JSON](https://www.json.org/). json optional object JSON-specific decoding options. lossy optional boolean Determines whether or not to replace invalid UTF-8 sequences instead of failing. When true, invalid UTF-8 sequences are replaced with the [`U+FFFD REPLACEMENT CHARACTER`](https://en.wikipedia.org/wiki/Specials_\(Unicode_block\)#Replacement_character). default: truecodec required json Decodes the raw bytes as [JSON](https://www.json.org/). Protobuf optional Decodes the raw bytes as [protobuf](https://protobuf.dev/). protobuf optional object Protobuf-specific decoding options. desc_file required string Path to desc file message_type required string message type. e.g package.message codec required protobuf Decodes the raw bytes as [protobuf](https://protobuf.dev/). Syslog optional Decodes the raw bytes as a Syslog message. Decodes either as the [RFC 3164](https://www.ietf.org/rfc/rfc3164.txt)-style format ("old" style) or the [RFC 5424](https://www.ietf.org/rfc/rfc5424.txt)-style format ("new" style, includes structured data). syslog optional object Syslog-specific decoding options. lossy optional boolean Determines whether or not to replace invalid UTF-8 sequences instead of failing. When true, invalid UTF-8 sequences are replaced with the [`U+FFFD REPLACEMENT CHARACTER`](https://en.wikipedia.org/wiki/Specials_\(Unicode_block\)#Replacement_character). default: truecodec required syslog Decodes the raw bytes as a Syslog message. Decodes either as the [RFC 3164](https://www.ietf.org/rfc/rfc3164.txt)-style format ("old" style) or the [RFC 5424](https://www.ietf.org/rfc/rfc5424.txt)-style format ("new" style, includes structured data). Native optional object Decodes the raw bytes as Vector's [native Protocol Buffers format](https://github.com/vectordotdev/vector/blob/master/lib/vector-core/proto/event.proto). This codec is **[experimental](https://vector.dev/highlights/2022-03-31-native-event-codecs)**. codec required native Decodes the raw bytes as Vector's [native Protocol Buffers format](https://github.com/vectordotdev/vector/blob/master/lib/vector-core/proto/event.proto). This codec is **[experimental](https://vector.dev/highlights/2022-03-31-native-event-codecs)**. NativeJson optional Decodes the raw bytes as Vector's [native JSON format](https://github.com/vectordotdev/vector/blob/master/lib/codecs/tests/data/native_encoding/schema.cue). This codec is **[experimental](https://vector.dev/highlights/2022-03-31-native-event-codecs)**. native_json optional object Vector's native JSON-specific decoding options. lossy optional boolean Determines whether or not to replace invalid UTF-8 sequences instead of failing. When true, invalid UTF-8 sequences are replaced with the [`U+FFFD REPLACEMENT CHARACTER`](https://en.wikipedia.org/wiki/Specials_\(Unicode_block\)#Replacement_character). default: truecodec required native_json Decodes the raw bytes as Vector's [native JSON format](https://github.com/vectordotdev/vector/blob/master/lib/codecs/tests/data/native_encoding/schema.cue). This codec is **[experimental](https://vector.dev/highlights/2022-03-31-native-event-codecs)**. Gelf optional Decodes the raw bytes as a [GELF](https://docs.graylog.org/docs/gelf) message. gelf optional object GELF-specific decoding options. lossy optional boolean Determines whether or not to replace invalid UTF-8 sequences instead of failing. When true, invalid UTF-8 sequences are replaced with the [`U+FFFD REPLACEMENT CHARACTER`](https://en.wikipedia.org/wiki/Specials_\(Unicode_block\)#Replacement_character). default: truecodec required gelf Decodes the raw bytes as a [GELF](https://docs.graylog.org/docs/gelf) message. encoding optional The expected encoding of received data. For `json` and `ndjson` encodings, the fields of the JSON objects are output as separate fields. Option 1 optional Content encoding. text optional text Plaintext. ndjson optional ndjson Newline-delimited JSON. json optional json JSON. binary optional binary Binary. framing optional Framing configuration. Framing handles how events are separated when encoded in a raw byte form, where each event is a frame that must be prefixed, or delimited, in a way that marks where an event begins and ends within the byte stream. Option 1 optional Framing configuration. Framing handles how events are separated when encoded in a raw byte form, where each event is a frame that must be prefixed, or delimited, in a way that marks where an event begins and ends within the byte stream. Bytes optional object Byte frames are passed through as-is according to the underlying I/O boundaries (for example, split between messages or stream segments). method required bytes Byte frames are passed through as-is according to the underlying I/O boundaries (for example, split between messages or stream segments). CharacterDelimited optional Byte frames which are delimited by a chosen character. character_delimited required object Options for the character delimited decoder. delimiter required integer The character that delimits byte sequences. max_length optional integer,​null The maximum length of the byte buffer. This length does *not* include the trailing delimiter. By default, there is no maximum length enforced. If events are malformed, this can lead to additional resource usage as events continue to be buffered in memory, and can potentially lead to memory exhaustion in extreme cases. If there is a risk of processing malformed data, such as logs with user-controlled input, consider setting the maximum length to a reasonably large value as a safety net. This ensures that processing is not actually unbounded. method required character_delimited Byte frames which are delimited by a chosen character. LengthDelimited optional object Byte frames which are prefixed by an unsigned big-endian 32-bit integer indicating the length. method required length_delimited Byte frames which are prefixed by an unsigned big-endian 32-bit integer indicating the length. NewlineDelimited optional Byte frames which are delimited by a newline character. newline_delimited optional object Options for the newline delimited decoder. max_length optional integer,​null The maximum length of the byte buffer. This length does *not* include the trailing delimiter. By default, there is no maximum length enforced. If events are malformed, this can lead to additional resource usage as events continue to be buffered in memory, and can potentially lead to memory exhaustion in extreme cases. If there is a risk of processing malformed data, such as logs with user-controlled input, consider setting the maximum length to a reasonably large value as a safety net. This ensures that processing is not actually unbounded. method required newline_delimited Byte frames which are delimited by a newline character. OctetCounting optional Byte frames according to the [octet counting](https://tools.ietf.org/html/rfc6587#section-3.4.1) format. octet_counting optional object Options for the octet counting decoder. max_length optional integer,​null The maximum length of the byte buffer. method required octet_counting Byte frames according to the [octet counting](https://tools.ietf.org/html/rfc6587#section-3.4.1) format. headers optional [string] A list of HTTP headers to include in the log event. These override any values included in the JSON payload with conflicting names. log_namespace optional boolean,​null The namespace to use for logs. This overrides the global setting. method optional Specifies the action of the HTTP request. HEAD optional HEAD HTTP HEAD method. GET optional GET HTTP GET method. POST optional POST HTTP POST method. PUT optional PUT HTTP Put method. PATCH optional PATCH HTTP PATCH method. DELETE optional DELETE HTTP DELETE method. path optional string The URL path on which log event POST requests are sent. path_key optional string The event key in which the requested URL path used to send the request is stored. query_parameters optional [string] A list of URL query parameters to include in the log event. These override any values included in the body with conflicting names. response_code optional integer Specifies the HTTP response status code that will be returned on successful requests. strict_path optional boolean Whether or not to treat the configured `path` as an absolute path. If set to `true`, only requests using the exact URL path specified in `path` are accepted. Otherwise, requests sent to a URL path that starts with the value of `path` are accepted. With `strict_path` set to `false` and `path` set to `""`, the configured HTTP source accepts requests from any URL path. tls optional Configures the TLS options for incoming/outgoing connections. Option 1 optional Configures the TLS options for incoming/outgoing connections. enabled optional boolean,​null Whether or not to require TLS for incoming or outgoing connections. When enabled and used for incoming connections, an identity certificate is also required. See `tls.crt_file` for more information. alpn_protocols optional array,​null Sets the list of supported ALPN protocols. Declare the supported ALPN protocols, which are used during negotiation with peer. They are prioritized in the order that they are defined. ca_file optional Absolute path to an additional CA certificate file. The certificate must be in the DER or PEM (X.509) format. Additionally, the certificate can be provided as an inline string in PEM format. Option 1 optional string A file path. crt_file optional Absolute path to a certificate file used to identify this server. The certificate must be in DER, PEM (X.509), or PKCS#12 format. Additionally, the certificate can be provided as an inline string in PEM format. If this is set, and is not a PKCS#12 archive, `key_file` must also be set. Option 1 optional string A file path. key_file optional Absolute path to a private key file used to identify this server. The key must be in DER or PEM (PKCS#8) format. Additionally, the key can be provided as an inline string in PEM format. Option 1 optional string A file path. key_pass optional string,​null Passphrase used to unlock the encrypted key file. This has no effect unless `key_file` is set. verify_certificate optional boolean,​null Enables certificate verification. If enabled, certificates must not be expired and must be issued by a trusted issuer. This verification operates in a hierarchical manner, checking that the leaf certificate (the certificate presented by the client/server) is not only valid, but that the issuer of that certificate is also valid, and so on until the verification process reaches a root certificate. Relevant for both incoming and outgoing connections. Do NOT set this to `false` unless you understand the risks of not verifying the validity of certificates. verify_hostname optional boolean,​null Enables hostname verification. If enabled, the hostname used to connect to the remote host must be present in the TLS certificate presented by the remote host, either as the Common Name or as an entry in the Subject Alternative Name extension. Only relevant for outgoing connections. Do NOT set this to `false` unless you understand the risks of not verifying the remote hostname. {% /tab %} {% tab title="simplehttp-request-example" %} ```yaml acknowledgements: enabled: null address: string auth: '' decoding: '' encoding: '' framing: '' headers: [] log_namespace: boolean method: POST path: / path_key: path query_parameters: [] response_code: 200 strict_path: true tls: '' type: http_server ``` {% /tab %} ### Internal Logs{% #internallogs %} Configuration for the `internal_logs` source. OptionsSchema {% tab title="internallogs-request-model" %} FieldrequiredTypeDescriptionhost_key optional string Overrides the name of the log field used to add the current hostname to each event. By default, the [global `log_schema.host_key` option](https://vector.dev/docs/reference/configuration/global-options/#log_schema.host_key) is used. Set to `""` to suppress this key. log_namespace optional boolean,​null The namespace to use for logs. This overrides the global setting. pid_key optional string Overrides the name of the log field used to add the current process ID to each event. By default, `"pid"` is used. Set to `""` to suppress this key. {% /tab %} {% tab title="internallogs-request-example" %} ```yaml host_key: host log_namespace: boolean pid_key: pid type: internal_logs ``` {% /tab %} ### Internal Metrics{% #internalmetrics %} Configuration for the `internal_metrics` source. OptionsSchema {% tab title="internalmetrics-request-model" %} FieldrequiredTypeDescriptionnamespace optional string Overrides the default namespace for the metrics emitted by the source. scrape_interval_secs optional number The interval between metric gathering, in seconds. tags optional object Tag configuration for the `internal_metrics` source. host_key optional string Overrides the name of the tag used to add the peer host to each metric. The value is the peer host's address, including the port. For example, `1.2.3.4:9000`. By default, the [global `log_schema.host_key` option](https://vector.dev/docs/reference/configuration/global-options/#log_schema.host_key) is used. Set to `""` to suppress this key. default: hostpid_key optional string,​null Sets the name of the tag to use to add the current process ID to each metric. By default, this is not set and the tag is not automatically added. default: null {% /tab %} {% tab title="internalmetrics-request-example" %} ```yaml namespace: vector scrape_interval_secs: 1 tags: host_key: host pid_key: null type: internal_metrics ``` {% /tab %} ### Kafka{% #kafkasource %} Configuration for the `kafka` source. OptionsSchema {% tab title="kafkasource-request-model" %} FieldrequiredTypeDescription acknowledgements optional object Controls how acknowledgements are handled by this source. **DEPRECATED**: This setting is **deprecated** in favor of enabling `acknowledgements` at the [global](https://vector.dev/docs/reference/configuration/global-options/#acknowledgements) or sink level. Enabling or disabling acknowledgements at the source level has **no effect** on acknowledgement behavior. See [End-to-end Acknowledgements](https://vector.dev/docs/about/under-the-hood/architecture/end-to-end-acknowledgements/) for more information on how event acknowledgement is handled. enabled optional boolean,​null Whether or not end-to-end acknowledgements are enabled for this source. default: nullauto_offset_reset optional string If offsets for consumer group do not exist, set them using this strategy. See the [librdkafka documentation](https://github.com/edenhill/librdkafka/blob/master/CONFIGURATION.md) for the `auto.offset.reset` option for further clarification. bootstrap_servers required string A comma-separated list of Kafka bootstrap servers. These are the servers in a Kafka cluster that a client should use to bootstrap its connection to the cluster, allowing discovery of all the other hosts in the cluster. Must be in the form of `host:port`, and comma-separated. commit_interval_ms optional integer The frequency that the consumer offsets are committed (written) to offset storage. decoding optional Configures how events are decoded from raw bytes. Bytes optional object Uses the raw bytes as-is. codec required bytes Uses the raw bytes as-is. Json optional Decodes the raw bytes as [JSON](https://www.json.org/). json optional object JSON-specific decoding options. lossy optional boolean Determines whether or not to replace invalid UTF-8 sequences instead of failing. When true, invalid UTF-8 sequences are replaced with the [`U+FFFD REPLACEMENT CHARACTER`](https://en.wikipedia.org/wiki/Specials_\(Unicode_block\)#Replacement_character). default: truecodec required json Decodes the raw bytes as [JSON](https://www.json.org/). Protobuf optional Decodes the raw bytes as [protobuf](https://protobuf.dev/). protobuf optional object Protobuf-specific decoding options. desc_file required string Path to desc file message_type required string message type. e.g package.message codec required protobuf Decodes the raw bytes as [protobuf](https://protobuf.dev/). Syslog optional Decodes the raw bytes as a Syslog message. Decodes either as the [RFC 3164](https://www.ietf.org/rfc/rfc3164.txt)-style format ("old" style) or the [RFC 5424](https://www.ietf.org/rfc/rfc5424.txt)-style format ("new" style, includes structured data). syslog optional object Syslog-specific decoding options. lossy optional boolean Determines whether or not to replace invalid UTF-8 sequences instead of failing. When true, invalid UTF-8 sequences are replaced with the [`U+FFFD REPLACEMENT CHARACTER`](https://en.wikipedia.org/wiki/Specials_\(Unicode_block\)#Replacement_character). default: truecodec required syslog Decodes the raw bytes as a Syslog message. Decodes either as the [RFC 3164](https://www.ietf.org/rfc/rfc3164.txt)-style format ("old" style) or the [RFC 5424](https://www.ietf.org/rfc/rfc5424.txt)-style format ("new" style, includes structured data). Native optional object Decodes the raw bytes as Vector's [native Protocol Buffers format](https://github.com/vectordotdev/vector/blob/master/lib/vector-core/proto/event.proto). This codec is **[experimental](https://vector.dev/highlights/2022-03-31-native-event-codecs)**. codec required native Decodes the raw bytes as Vector's [native Protocol Buffers format](https://github.com/vectordotdev/vector/blob/master/lib/vector-core/proto/event.proto). This codec is **[experimental](https://vector.dev/highlights/2022-03-31-native-event-codecs)**. NativeJson optional Decodes the raw bytes as Vector's [native JSON format](https://github.com/vectordotdev/vector/blob/master/lib/codecs/tests/data/native_encoding/schema.cue). This codec is **[experimental](https://vector.dev/highlights/2022-03-31-native-event-codecs)**. native_json optional object Vector's native JSON-specific decoding options. lossy optional boolean Determines whether or not to replace invalid UTF-8 sequences instead of failing. When true, invalid UTF-8 sequences are replaced with the [`U+FFFD REPLACEMENT CHARACTER`](https://en.wikipedia.org/wiki/Specials_\(Unicode_block\)#Replacement_character). default: truecodec required native_json Decodes the raw bytes as Vector's [native JSON format](https://github.com/vectordotdev/vector/blob/master/lib/codecs/tests/data/native_encoding/schema.cue). This codec is **[experimental](https://vector.dev/highlights/2022-03-31-native-event-codecs)**. Gelf optional Decodes the raw bytes as a [GELF](https://docs.graylog.org/docs/gelf) message. gelf optional object GELF-specific decoding options. lossy optional boolean Determines whether or not to replace invalid UTF-8 sequences instead of failing. When true, invalid UTF-8 sequences are replaced with the [`U+FFFD REPLACEMENT CHARACTER`](https://en.wikipedia.org/wiki/Specials_\(Unicode_block\)#Replacement_character). default: truecodec required gelf Decodes the raw bytes as a [GELF](https://docs.graylog.org/docs/gelf) message. fetch_wait_max_ms optional integer Maximum time the broker may wait to fill the response. framing optional Framing configuration. Framing handles how events are separated when encoded in a raw byte form, where each event is a frame that must be prefixed, or delimited, in a way that marks where an event begins and ends within the byte stream. Bytes optional object Byte frames are passed through as-is according to the underlying I/O boundaries (for example, split between messages or stream segments). method required bytes Byte frames are passed through as-is according to the underlying I/O boundaries (for example, split between messages or stream segments). CharacterDelimited optional Byte frames which are delimited by a chosen character. character_delimited required object Options for the character delimited decoder. delimiter required integer The character that delimits byte sequences. max_length optional integer,​null The maximum length of the byte buffer. This length does *not* include the trailing delimiter. By default, there is no maximum length enforced. If events are malformed, this can lead to additional resource usage as events continue to be buffered in memory, and can potentially lead to memory exhaustion in extreme cases. If there is a risk of processing malformed data, such as logs with user-controlled input, consider setting the maximum length to a reasonably large value as a safety net. This ensures that processing is not actually unbounded. method required character_delimited Byte frames which are delimited by a chosen character. LengthDelimited optional object Byte frames which are prefixed by an unsigned big-endian 32-bit integer indicating the length. method required length_delimited Byte frames which are prefixed by an unsigned big-endian 32-bit integer indicating the length. NewlineDelimited optional Byte frames which are delimited by a newline character. newline_delimited optional object Options for the newline delimited decoder. max_length optional integer,​null The maximum length of the byte buffer. This length does *not* include the trailing delimiter. By default, there is no maximum length enforced. If events are malformed, this can lead to additional resource usage as events continue to be buffered in memory, and can potentially lead to memory exhaustion in extreme cases. If there is a risk of processing malformed data, such as logs with user-controlled input, consider setting the maximum length to a reasonably large value as a safety net. This ensures that processing is not actually unbounded. method required newline_delimited Byte frames which are delimited by a newline character. OctetCounting optional Byte frames according to the [octet counting](https://tools.ietf.org/html/rfc6587#section-3.4.1) format. octet_counting optional object Options for the octet counting decoder. max_length optional integer,​null The maximum length of the byte buffer. method required octet_counting Byte frames according to the [octet counting](https://tools.ietf.org/html/rfc6587#section-3.4.1) format. group_id required string The consumer group name to be used to consume events from Kafka. headers_key optional string Overrides the name of the log field used to add the headers to each event. The value is the headers of the Kafka message itself. By default, `"headers"` is used. key_field optional string Overrides the name of the log field used to add the message key to each event. The value is the message key of the Kafka message itself. By default, `"message_key"` is used. librdkafka_options optional object,​null Advanced options set directly on the underlying `librdkafka` client. See the [librdkafka documentation](https://github.com/edenhill/librdkafka/blob/master/CONFIGURATION.md) for details. log_namespace optional boolean,​null The namespace to use for logs. This overrides the global setting. metrics optional object Metrics configuration. topic_lag_metric required boolean Expose topic lag metrics for all topics and partitions. Metric names are `kafka_consumer_lag`. default: falseoffset_key optional string Overrides the name of the log field used to add the offset to each event. The value is the offset of the Kafka message itself. By default, `"offset"` is used. partition_key optional string Overrides the name of the log field used to add the partition to each event. The value is the partition from which the Kafka message was consumed from. By default, `"partition"` is used. session_timeout_ms optional integer The Kafka session timeout. socket_timeout_ms optional integer Timeout for network requests. topic_key optional string Overrides the name of the log field used to add the topic to each event. The value is the topic from which the Kafka message was consumed from. By default, `"topic"` is used. topics required [string] The Kafka topics names to read events from. Regular expression syntax is supported if the topic begins with `^`. sasl optional Configuration for SASL authentication when interacting with Kafka. Option 1 optional object Configuration for SASL authentication when interacting with Kafka. enabled optional boolean,​null Enables SASL authentication. Only `PLAIN`- and `SCRAM`-based mechanisms are supported when configuring SASL authentication using `sasl.*`. For other mechanisms, `librdkafka_options.*` must be used directly to configure other `librdkafka`-specific values. If using `sasl.kerberos.*` as an example, where `*` is `service.name`, `principal`, `kinit.md`, etc., then `librdkafka_options.*` as a result becomes `librdkafka_options.sasl.kerberos.service.name`, `librdkafka_options.sasl.kerberos.principal`, etc. See the [librdkafka documentation](https://github.com/edenhill/librdkafka/blob/master/CONFIGURATION.md) for details. SASL authentication is not supported on Windows. mechanism optional string,​null The SASL mechanism to use. password optional The SASL password. Option 1 optional string Wrapper for sensitive strings containing credentials username optional string,​null The SASL username. tls optional Configures the TLS options for incoming/outgoing connections. Option 1 optional Configures the TLS options for incoming/outgoing connections. enabled optional boolean,​null Whether or not to require TLS for incoming or outgoing connections. When enabled and used for incoming connections, an identity certificate is also required. See `tls.crt_file` for more information. alpn_protocols optional array,​null Sets the list of supported ALPN protocols. Declare the supported ALPN protocols, which are used during negotiation with peer. They are prioritized in the order that they are defined. ca_file optional Absolute path to an additional CA certificate file. The certificate must be in the DER or PEM (X.509) format. Additionally, the certificate can be provided as an inline string in PEM format. Option 1 optional string A file path. crt_file optional Absolute path to a certificate file used to identify this server. The certificate must be in DER, PEM (X.509), or PKCS#12 format. Additionally, the certificate can be provided as an inline string in PEM format. If this is set, and is not a PKCS#12 archive, `key_file` must also be set. Option 1 optional string A file path. key_file optional Absolute path to a private key file used to identify this server. The key must be in DER or PEM (PKCS#8) format. Additionally, the key can be provided as an inline string in PEM format. Option 1 optional string A file path. key_pass optional string,​null Passphrase used to unlock the encrypted key file. This has no effect unless `key_file` is set. verify_certificate optional boolean,​null Enables certificate verification. If enabled, certificates must not be expired and must be issued by a trusted issuer. This verification operates in a hierarchical manner, checking that the leaf certificate (the certificate presented by the client/server) is not only valid, but that the issuer of that certificate is also valid, and so on until the verification process reaches a root certificate. Relevant for both incoming and outgoing connections. Do NOT set this to `false` unless you understand the risks of not verifying the validity of certificates. verify_hostname optional boolean,​null Enables hostname verification. If enabled, the hostname used to connect to the remote host must be present in the TLS certificate presented by the remote host, either as the Common Name or as an entry in the Subject Alternative Name extension. Only relevant for outgoing connections. Do NOT set this to `false` unless you understand the risks of not verifying the remote hostname. {% /tab %} {% tab title="kafkasource-request-example" %} ```yaml acknowledgements: enabled: null auto_offset_reset: largest bootstrap_servers: string commit_interval_ms: 5000 decoding: codec: bytes fetch_wait_max_ms: 100 framing: method: bytes group_id: string headers_key: headers key_field: message_key librdkafka_options: object log_namespace: boolean metrics: topic_lag_metric: false offset_key: offset partition_key: partition session_timeout_ms: 10000 socket_timeout_ms: 60000 topic_key: topic topics: array type: kafka ``` {% /tab %} ### Logstash{% #logstash %} Configuration for the `logstash` source. OptionsSchema {% tab title="logstash-request-model" %} FieldrequiredTypeDescription acknowledgements optional object Controls how acknowledgements are handled by this source. **DEPRECATED**: This setting is **deprecated** in favor of enabling `acknowledgements` at the [global](https://vector.dev/docs/reference/configuration/global-options/#acknowledgements) or sink level. Enabling or disabling acknowledgements at the source level has **no effect** on acknowledgement behavior. See [End-to-end Acknowledgements](https://vector.dev/docs/about/under-the-hood/architecture/end-to-end-acknowledgements/) for more information on how event acknowledgement is handled. enabled optional boolean,​null Whether or not end-to-end acknowledgements are enabled for this source. default: nulladdress required string The socket address to listen for connections on, or `systemd{#N}` to use the Nth socket passed by systemd socket activation. If a socket address is used, it *must* include a port. connection_limit optional integer,​null The maximum number of TCP connections that are allowed at any given time. keepalive optional TCP keepalive settings for socket-based components. Option 1 optional object TCP keepalive settings for socket-based components. time_secs optional integer,​null The time to wait before starting to send TCP keepalive probes on an idle connection. log_namespace optional boolean,​null The namespace to use for logs. This overrides the global setting. receive_buffer_bytes optional integer,​null The size of the receive buffer used for each connection. tls optional TlsEnableableConfig for `sources`, adding metadata from the client certificate. Option 1 optional TlsEnableableConfig for `sources`, adding metadata from the client certificate. client_metadata_key optional Event field for client certificate metadata. Option 1 optional string An optional path that deserializes an empty string to `None`. enabled optional boolean,​null Whether or not to require TLS for incoming or outgoing connections. When enabled and used for incoming connections, an identity certificate is also required. See `tls.crt_file` for more information. alpn_protocols optional array,​null Sets the list of supported ALPN protocols. Declare the supported ALPN protocols, which are used during negotiation with peer. They are prioritized in the order that they are defined. ca_file optional Absolute path to an additional CA certificate file. The certificate must be in the DER or PEM (X.509) format. Additionally, the certificate can be provided as an inline string in PEM format. Option 1 optional string A file path. crt_file optional Absolute path to a certificate file used to identify this server. The certificate must be in DER, PEM (X.509), or PKCS#12 format. Additionally, the certificate can be provided as an inline string in PEM format. If this is set, and is not a PKCS#12 archive, `key_file` must also be set. Option 1 optional string A file path. key_file optional Absolute path to a private key file used to identify this server. The key must be in DER or PEM (PKCS#8) format. Additionally, the key can be provided as an inline string in PEM format. Option 1 optional string A file path. key_pass optional string,​null Passphrase used to unlock the encrypted key file. This has no effect unless `key_file` is set. verify_certificate optional boolean,​null Enables certificate verification. If enabled, certificates must not be expired and must be issued by a trusted issuer. This verification operates in a hierarchical manner, checking that the leaf certificate (the certificate presented by the client/server) is not only valid, but that the issuer of that certificate is also valid, and so on until the verification process reaches a root certificate. Relevant for both incoming and outgoing connections. Do NOT set this to `false` unless you understand the risks of not verifying the validity of certificates. verify_hostname optional boolean,​null Enables hostname verification. If enabled, the hostname used to connect to the remote host must be present in the TLS certificate presented by the remote host, either as the Common Name or as an entry in the Subject Alternative Name extension. Only relevant for outgoing connections. Do NOT set this to `false` unless you understand the risks of not verifying the remote hostname. {% /tab %} {% tab title="logstash-request-example" %} ```yaml acknowledgements: enabled: null address: string connection_limit: integer keepalive: '' log_namespace: boolean receive_buffer_bytes: integer tls: '' type: logstash ``` {% /tab %} ### NATS{% #natssource %} Configuration for the `nats` source. OptionsSchema {% tab title="natssource-request-model" %} FieldrequiredTypeDescription auth optional Configuration of the authentication strategy when interacting with NATS. Option 1 optional Configuration of the authentication strategy when interacting with NATS. UserPassword optional object Username/password authentication. strategy required user_password Username/password authentication. user_password required object Username and password configuration. password required string Password. user required string Username. Token optional object Token authentication. strategy required token Token authentication. token required object Token configuration. value required string Token. CredentialsFile optional object Credentials file authentication. (JWT-based) credentials_file required object Credentials file configuration. path required string Path to credentials file. strategy required credentials_file Credentials file authentication. (JWT-based) Nkey optional object NKey authentication. nkey required object NKeys configuration. nkey required string User. Conceptually, this is equivalent to a public key. seed required string Seed. Conceptually, this is equivalent to a private key. strategy required nkey NKey authentication. connection_name required string A [name](https://docs.nats.io/using-nats/developer/connecting/name) assigned to the NATS connection. decoding optional Configures how events are decoded from raw bytes. Bytes optional object Uses the raw bytes as-is. codec required bytes Uses the raw bytes as-is. Json optional Decodes the raw bytes as [JSON](https://www.json.org/). json optional object JSON-specific decoding options. lossy optional boolean Determines whether or not to replace invalid UTF-8 sequences instead of failing. When true, invalid UTF-8 sequences are replaced with the [`U+FFFD REPLACEMENT CHARACTER`](https://en.wikipedia.org/wiki/Specials_\(Unicode_block\)#Replacement_character). default: truecodec required json Decodes the raw bytes as [JSON](https://www.json.org/). Protobuf optional Decodes the raw bytes as [protobuf](https://protobuf.dev/). protobuf optional object Protobuf-specific decoding options. desc_file required string Path to desc file message_type required string message type. e.g package.message codec required protobuf Decodes the raw bytes as [protobuf](https://protobuf.dev/). Syslog optional Decodes the raw bytes as a Syslog message. Decodes either as the [RFC 3164](https://www.ietf.org/rfc/rfc3164.txt)-style format ("old" style) or the [RFC 5424](https://www.ietf.org/rfc/rfc5424.txt)-style format ("new" style, includes structured data). syslog optional object Syslog-specific decoding options. lossy optional boolean Determines whether or not to replace invalid UTF-8 sequences instead of failing. When true, invalid UTF-8 sequences are replaced with the [`U+FFFD REPLACEMENT CHARACTER`](https://en.wikipedia.org/wiki/Specials_\(Unicode_block\)#Replacement_character). default: truecodec required syslog Decodes the raw bytes as a Syslog message. Decodes either as the [RFC 3164](https://www.ietf.org/rfc/rfc3164.txt)-style format ("old" style) or the [RFC 5424](https://www.ietf.org/rfc/rfc5424.txt)-style format ("new" style, includes structured data). Native optional object Decodes the raw bytes as Vector's [native Protocol Buffers format](https://github.com/vectordotdev/vector/blob/master/lib/vector-core/proto/event.proto). This codec is **[experimental](https://vector.dev/highlights/2022-03-31-native-event-codecs)**. codec required native Decodes the raw bytes as Vector's [native Protocol Buffers format](https://github.com/vectordotdev/vector/blob/master/lib/vector-core/proto/event.proto). This codec is **[experimental](https://vector.dev/highlights/2022-03-31-native-event-codecs)**. NativeJson optional Decodes the raw bytes as Vector's [native JSON format](https://github.com/vectordotdev/vector/blob/master/lib/codecs/tests/data/native_encoding/schema.cue). This codec is **[experimental](https://vector.dev/highlights/2022-03-31-native-event-codecs)**. native_json optional object Vector's native JSON-specific decoding options. lossy optional boolean Determines whether or not to replace invalid UTF-8 sequences instead of failing. When true, invalid UTF-8 sequences are replaced with the [`U+FFFD REPLACEMENT CHARACTER`](https://en.wikipedia.org/wiki/Specials_\(Unicode_block\)#Replacement_character). default: truecodec required native_json Decodes the raw bytes as Vector's [native JSON format](https://github.com/vectordotdev/vector/blob/master/lib/codecs/tests/data/native_encoding/schema.cue). This codec is **[experimental](https://vector.dev/highlights/2022-03-31-native-event-codecs)**. Gelf optional Decodes the raw bytes as a [GELF](https://docs.graylog.org/docs/gelf) message. gelf optional object GELF-specific decoding options. lossy optional boolean Determines whether or not to replace invalid UTF-8 sequences instead of failing. When true, invalid UTF-8 sequences are replaced with the [`U+FFFD REPLACEMENT CHARACTER`](https://en.wikipedia.org/wiki/Specials_\(Unicode_block\)#Replacement_character). default: truecodec required gelf Decodes the raw bytes as a [GELF](https://docs.graylog.org/docs/gelf) message. framing optional Framing configuration. Framing handles how events are separated when encoded in a raw byte form, where each event is a frame that must be prefixed, or delimited, in a way that marks where an event begins and ends within the byte stream. Bytes optional object Byte frames are passed through as-is according to the underlying I/O boundaries (for example, split between messages or stream segments). method required bytes Byte frames are passed through as-is according to the underlying I/O boundaries (for example, split between messages or stream segments). CharacterDelimited optional Byte frames which are delimited by a chosen character. character_delimited required object Options for the character delimited decoder. delimiter required integer The character that delimits byte sequences. max_length optional integer,​null The maximum length of the byte buffer. This length does *not* include the trailing delimiter. By default, there is no maximum length enforced. If events are malformed, this can lead to additional resource usage as events continue to be buffered in memory, and can potentially lead to memory exhaustion in extreme cases. If there is a risk of processing malformed data, such as logs with user-controlled input, consider setting the maximum length to a reasonably large value as a safety net. This ensures that processing is not actually unbounded. method required character_delimited Byte frames which are delimited by a chosen character. LengthDelimited optional object Byte frames which are prefixed by an unsigned big-endian 32-bit integer indicating the length. method required length_delimited Byte frames which are prefixed by an unsigned big-endian 32-bit integer indicating the length. NewlineDelimited optional Byte frames which are delimited by a newline character. newline_delimited optional object Options for the newline delimited decoder. max_length optional integer,​null The maximum length of the byte buffer. This length does *not* include the trailing delimiter. By default, there is no maximum length enforced. If events are malformed, this can lead to additional resource usage as events continue to be buffered in memory, and can potentially lead to memory exhaustion in extreme cases. If there is a risk of processing malformed data, such as logs with user-controlled input, consider setting the maximum length to a reasonably large value as a safety net. This ensures that processing is not actually unbounded. method required newline_delimited Byte frames which are delimited by a newline character. OctetCounting optional Byte frames according to the [octet counting](https://tools.ietf.org/html/rfc6587#section-3.4.1) format. octet_counting optional object Options for the octet counting decoder. max_length optional integer,​null The maximum length of the byte buffer. method required octet_counting Byte frames according to the [octet counting](https://tools.ietf.org/html/rfc6587#section-3.4.1) format. log_namespace optional boolean,​null The namespace to use for logs. This overrides the global setting. queue optional string,​null The NATS queue group to join. subject required string The NATS [subject](https://docs.nats.io/nats-concepts/subjects) to pull messages from. subject_key_field optional string The `NATS` subject key. tls optional Configures the TLS options for incoming/outgoing connections. Option 1 optional Configures the TLS options for incoming/outgoing connections. enabled optional boolean,​null Whether or not to require TLS for incoming or outgoing connections. When enabled and used for incoming connections, an identity certificate is also required. See `tls.crt_file` for more information. alpn_protocols optional array,​null Sets the list of supported ALPN protocols. Declare the supported ALPN protocols, which are used during negotiation with peer. They are prioritized in the order that they are defined. ca_file optional Absolute path to an additional CA certificate file. The certificate must be in the DER or PEM (X.509) format. Additionally, the certificate can be provided as an inline string in PEM format. Option 1 optional string A file path. crt_file optional Absolute path to a certificate file used to identify this server. The certificate must be in DER, PEM (X.509), or PKCS#12 format. Additionally, the certificate can be provided as an inline string in PEM format. If this is set, and is not a PKCS#12 archive, `key_file` must also be set. Option 1 optional string A file path. key_file optional Absolute path to a private key file used to identify this server. The key must be in DER or PEM (PKCS#8) format. Additionally, the key can be provided as an inline string in PEM format. Option 1 optional string A file path. key_pass optional string,​null Passphrase used to unlock the encrypted key file. This has no effect unless `key_file` is set. verify_certificate optional boolean,​null Enables certificate verification. If enabled, certificates must not be expired and must be issued by a trusted issuer. This verification operates in a hierarchical manner, checking that the leaf certificate (the certificate presented by the client/server) is not only valid, but that the issuer of that certificate is also valid, and so on until the verification process reaches a root certificate. Relevant for both incoming and outgoing connections. Do NOT set this to `false` unless you understand the risks of not verifying the validity of certificates. verify_hostname optional boolean,​null Enables hostname verification. If enabled, the hostname used to connect to the remote host must be present in the TLS certificate presented by the remote host, either as the Common Name or as an entry in the Subject Alternative Name extension. Only relevant for outgoing connections. Do NOT set this to `false` unless you understand the risks of not verifying the remote hostname. url required string The NATS URL to connect to. The URL takes the form of `nats://server:port`. If the port is not specified it defaults to 4222. {% /tab %} {% tab title="natssource-request-example" %} ```yaml auth: '' connection_name: string decoding: codec: bytes framing: method: bytes log_namespace: boolean queue: string subject: string subject_key_field: subject tls: '' url: string type: nats ``` {% /tab %} ### OpenTelemetry{% #opentelemetry %} Configuration for the `opentelemetry` source. OptionsSchema {% tab title="opentelemetry-request-model" %} FieldrequiredTypeDescription acknowledgements optional object Controls how acknowledgements are handled by this source. **DEPRECATED**: This setting is **deprecated** in favor of enabling `acknowledgements` at the [global](https://vector.dev/docs/reference/configuration/global-options/#acknowledgements) or sink level. Enabling or disabling acknowledgements at the source level has **no effect** on acknowledgement behavior. See [End-to-end Acknowledgements](https://vector.dev/docs/about/under-the-hood/architecture/end-to-end-acknowledgements/) for more information on how event acknowledgement is handled. enabled optional boolean,​null Whether or not end-to-end acknowledgements are enabled for this source. default: null grpc required object Configuration for the `opentelemetry` gRPC server. address required string The socket address to listen for connections on. It *must* include a port. tls optional Configures the TLS options for incoming/outgoing connections. Option 1 optional Configures the TLS options for incoming/outgoing connections. enabled optional boolean,​null Whether or not to require TLS for incoming or outgoing connections. When enabled and used for incoming connections, an identity certificate is also required. See `tls.crt_file` for more information. alpn_protocols optional array,​null Sets the list of supported ALPN protocols. Declare the supported ALPN protocols, which are used during negotiation with peer. They are prioritized in the order that they are defined. ca_file optional Absolute path to an additional CA certificate file. The certificate must be in the DER or PEM (X.509) format. Additionally, the certificate can be provided as an inline string in PEM format. Option 1 optional string A file path. crt_file optional Absolute path to a certificate file used to identify this server. The certificate must be in DER, PEM (X.509), or PKCS#12 format. Additionally, the certificate can be provided as an inline string in PEM format. If this is set, and is not a PKCS#12 archive, `key_file` must also be set. Option 1 optional string A file path. key_file optional Absolute path to a private key file used to identify this server. The key must be in DER or PEM (PKCS#8) format. Additionally, the key can be provided as an inline string in PEM format. Option 1 optional string A file path. key_pass optional string,​null Passphrase used to unlock the encrypted key file. This has no effect unless `key_file` is set. verify_certificate optional boolean,​null Enables certificate verification. If enabled, certificates must not be expired and must be issued by a trusted issuer. This verification operates in a hierarchical manner, checking that the leaf certificate (the certificate presented by the client/server) is not only valid, but that the issuer of that certificate is also valid, and so on until the verification process reaches a root certificate. Relevant for both incoming and outgoing connections. Do NOT set this to `false` unless you understand the risks of not verifying the validity of certificates. verify_hostname optional boolean,​null Enables hostname verification. If enabled, the hostname used to connect to the remote host must be present in the TLS certificate presented by the remote host, either as the Common Name or as an entry in the Subject Alternative Name extension. Only relevant for outgoing connections. Do NOT set this to `false` unless you understand the risks of not verifying the remote hostname. http required object Configuration for the `opentelemetry` HTTP server. address required string The socket address to listen for connections on. It *must* include a port. tls optional Configures the TLS options for incoming/outgoing connections. Option 1 optional Configures the TLS options for incoming/outgoing connections. enabled optional boolean,​null Whether or not to require TLS for incoming or outgoing connections. When enabled and used for incoming connections, an identity certificate is also required. See `tls.crt_file` for more information. alpn_protocols optional array,​null Sets the list of supported ALPN protocols. Declare the supported ALPN protocols, which are used during negotiation with peer. They are prioritized in the order that they are defined. ca_file optional Absolute path to an additional CA certificate file. The certificate must be in the DER or PEM (X.509) format. Additionally, the certificate can be provided as an inline string in PEM format. Option 1 optional string A file path. crt_file optional Absolute path to a certificate file used to identify this server. The certificate must be in DER, PEM (X.509), or PKCS#12 format. Additionally, the certificate can be provided as an inline string in PEM format. If this is set, and is not a PKCS#12 archive, `key_file` must also be set. Option 1 optional string A file path. key_file optional Absolute path to a private key file used to identify this server. The key must be in DER or PEM (PKCS#8) format. Additionally, the key can be provided as an inline string in PEM format. Option 1 optional string A file path. key_pass optional string,​null Passphrase used to unlock the encrypted key file. This has no effect unless `key_file` is set. verify_certificate optional boolean,​null Enables certificate verification. If enabled, certificates must not be expired and must be issued by a trusted issuer. This verification operates in a hierarchical manner, checking that the leaf certificate (the certificate presented by the client/server) is not only valid, but that the issuer of that certificate is also valid, and so on until the verification process reaches a root certificate. Relevant for both incoming and outgoing connections. Do NOT set this to `false` unless you understand the risks of not verifying the validity of certificates. verify_hostname optional boolean,​null Enables hostname verification. If enabled, the hostname used to connect to the remote host must be present in the TLS certificate presented by the remote host, either as the Common Name or as an entry in the Subject Alternative Name extension. Only relevant for outgoing connections. Do NOT set this to `false` unless you understand the risks of not verifying the remote hostname. log_namespace optional boolean,​null The namespace to use for logs. This overrides the global setting. {% /tab %} {% tab title="opentelemetry-request-example" %} ```yaml acknowledgements: enabled: null grpc: object http: object log_namespace: boolean type: opentelemetry ``` {% /tab %} ### Prometheus Remote Write{% #prometheusremotewrite %} Configuration for the `prometheus_remote_write` source. OptionsSchema {% tab title="prometheusremotewrite-request-model" %} FieldrequiredTypeDescription acknowledgements optional object Controls how acknowledgements are handled by this source. **DEPRECATED**: This setting is **deprecated** in favor of enabling `acknowledgements` at the [global](https://vector.dev/docs/reference/configuration/global-options/#acknowledgements) or sink level. Enabling or disabling acknowledgements at the source level has **no effect** on acknowledgement behavior. See [End-to-end Acknowledgements](https://vector.dev/docs/about/under-the-hood/architecture/end-to-end-acknowledgements/) for more information on how event acknowledgement is handled. enabled optional boolean,​null Whether or not end-to-end acknowledgements are enabled for this source. default: nulladdress required string The socket address to accept connections on. The address *must* include a port. auth optional HTTP Basic authentication configuration. Option 1 optional object HTTP Basic authentication configuration. password required string The password for basic authentication. username required string The username for basic authentication. tls optional Configures the TLS options for incoming/outgoing connections. Option 1 optional Configures the TLS options for incoming/outgoing connections. enabled optional boolean,​null Whether or not to require TLS for incoming or outgoing connections. When enabled and used for incoming connections, an identity certificate is also required. See `tls.crt_file` for more information. alpn_protocols optional array,​null Sets the list of supported ALPN protocols. Declare the supported ALPN protocols, which are used during negotiation with peer. They are prioritized in the order that they are defined. ca_file optional Absolute path to an additional CA certificate file. The certificate must be in the DER or PEM (X.509) format. Additionally, the certificate can be provided as an inline string in PEM format. Option 1 optional string A file path. crt_file optional Absolute path to a certificate file used to identify this server. The certificate must be in DER, PEM (X.509), or PKCS#12 format. Additionally, the certificate can be provided as an inline string in PEM format. If this is set, and is not a PKCS#12 archive, `key_file` must also be set. Option 1 optional string A file path. key_file optional Absolute path to a private key file used to identify this server. The key must be in DER or PEM (PKCS#8) format. Additionally, the key can be provided as an inline string in PEM format. Option 1 optional string A file path. key_pass optional string,​null Passphrase used to unlock the encrypted key file. This has no effect unless `key_file` is set. verify_certificate optional boolean,​null Enables certificate verification. If enabled, certificates must not be expired and must be issued by a trusted issuer. This verification operates in a hierarchical manner, checking that the leaf certificate (the certificate presented by the client/server) is not only valid, but that the issuer of that certificate is also valid, and so on until the verification process reaches a root certificate. Relevant for both incoming and outgoing connections. Do NOT set this to `false` unless you understand the risks of not verifying the validity of certificates. verify_hostname optional boolean,​null Enables hostname verification. If enabled, the hostname used to connect to the remote host must be present in the TLS certificate presented by the remote host, either as the Common Name or as an entry in the Subject Alternative Name extension. Only relevant for outgoing connections. Do NOT set this to `false` unless you understand the risks of not verifying the remote hostname. {% /tab %} {% tab title="prometheusremotewrite-request-example" %} ```yaml acknowledgements: enabled: null address: string auth: '' tls: '' type: prometheus_remote_write ``` {% /tab %} ### Redis{% #redissource %} Configuration for the `redis` source. OptionsSchema {% tab title="redissource-request-model" %} FieldrequiredTypeDescription data_type optional The Redis data type (`list` or `channel`) to use. list optional list The `list` data type. channel optional channel The `channel` data type. This is based on Redis' Pub/Sub capabilities. decoding optional Configures how events are decoded from raw bytes. Bytes optional object Uses the raw bytes as-is. codec required bytes Uses the raw bytes as-is. Json optional Decodes the raw bytes as [JSON](https://www.json.org/). json optional object JSON-specific decoding options. lossy optional boolean Determines whether or not to replace invalid UTF-8 sequences instead of failing. When true, invalid UTF-8 sequences are replaced with the [`U+FFFD REPLACEMENT CHARACTER`](https://en.wikipedia.org/wiki/Specials_\(Unicode_block\)#Replacement_character). default: truecodec required json Decodes the raw bytes as [JSON](https://www.json.org/). Protobuf optional Decodes the raw bytes as [protobuf](https://protobuf.dev/). protobuf optional object Protobuf-specific decoding options. desc_file required string Path to desc file message_type required string message type. e.g package.message codec required protobuf Decodes the raw bytes as [protobuf](https://protobuf.dev/). Syslog optional Decodes the raw bytes as a Syslog message. Decodes either as the [RFC 3164](https://www.ietf.org/rfc/rfc3164.txt)-style format ("old" style) or the [RFC 5424](https://www.ietf.org/rfc/rfc5424.txt)-style format ("new" style, includes structured data). syslog optional object Syslog-specific decoding options. lossy optional boolean Determines whether or not to replace invalid UTF-8 sequences instead of failing. When true, invalid UTF-8 sequences are replaced with the [`U+FFFD REPLACEMENT CHARACTER`](https://en.wikipedia.org/wiki/Specials_\(Unicode_block\)#Replacement_character). default: truecodec required syslog Decodes the raw bytes as a Syslog message. Decodes either as the [RFC 3164](https://www.ietf.org/rfc/rfc3164.txt)-style format ("old" style) or the [RFC 5424](https://www.ietf.org/rfc/rfc5424.txt)-style format ("new" style, includes structured data). Native optional object Decodes the raw bytes as Vector's [native Protocol Buffers format](https://github.com/vectordotdev/vector/blob/master/lib/vector-core/proto/event.proto). This codec is **[experimental](https://vector.dev/highlights/2022-03-31-native-event-codecs)**. codec required native Decodes the raw bytes as Vector's [native Protocol Buffers format](https://github.com/vectordotdev/vector/blob/master/lib/vector-core/proto/event.proto). This codec is **[experimental](https://vector.dev/highlights/2022-03-31-native-event-codecs)**. NativeJson optional Decodes the raw bytes as Vector's [native JSON format](https://github.com/vectordotdev/vector/blob/master/lib/codecs/tests/data/native_encoding/schema.cue). This codec is **[experimental](https://vector.dev/highlights/2022-03-31-native-event-codecs)**. native_json optional object Vector's native JSON-specific decoding options. lossy optional boolean Determines whether or not to replace invalid UTF-8 sequences instead of failing. When true, invalid UTF-8 sequences are replaced with the [`U+FFFD REPLACEMENT CHARACTER`](https://en.wikipedia.org/wiki/Specials_\(Unicode_block\)#Replacement_character). default: truecodec required native_json Decodes the raw bytes as Vector's [native JSON format](https://github.com/vectordotdev/vector/blob/master/lib/codecs/tests/data/native_encoding/schema.cue). This codec is **[experimental](https://vector.dev/highlights/2022-03-31-native-event-codecs)**. Gelf optional Decodes the raw bytes as a [GELF](https://docs.graylog.org/docs/gelf) message. gelf optional object GELF-specific decoding options. lossy optional boolean Determines whether or not to replace invalid UTF-8 sequences instead of failing. When true, invalid UTF-8 sequences are replaced with the [`U+FFFD REPLACEMENT CHARACTER`](https://en.wikipedia.org/wiki/Specials_\(Unicode_block\)#Replacement_character). default: truecodec required gelf Decodes the raw bytes as a [GELF](https://docs.graylog.org/docs/gelf) message. framing optional Framing configuration. Framing handles how events are separated when encoded in a raw byte form, where each event is a frame that must be prefixed, or delimited, in a way that marks where an event begins and ends within the byte stream. Bytes optional object Byte frames are passed through as-is according to the underlying I/O boundaries (for example, split between messages or stream segments). method required bytes Byte frames are passed through as-is according to the underlying I/O boundaries (for example, split between messages or stream segments). CharacterDelimited optional Byte frames which are delimited by a chosen character. character_delimited required object Options for the character delimited decoder. delimiter required integer The character that delimits byte sequences. max_length optional integer,​null The maximum length of the byte buffer. This length does *not* include the trailing delimiter. By default, there is no maximum length enforced. If events are malformed, this can lead to additional resource usage as events continue to be buffered in memory, and can potentially lead to memory exhaustion in extreme cases. If there is a risk of processing malformed data, such as logs with user-controlled input, consider setting the maximum length to a reasonably large value as a safety net. This ensures that processing is not actually unbounded. method required character_delimited Byte frames which are delimited by a chosen character. LengthDelimited optional object Byte frames which are prefixed by an unsigned big-endian 32-bit integer indicating the length. method required length_delimited Byte frames which are prefixed by an unsigned big-endian 32-bit integer indicating the length. NewlineDelimited optional Byte frames which are delimited by a newline character. newline_delimited optional object Options for the newline delimited decoder. max_length optional integer,​null The maximum length of the byte buffer. This length does *not* include the trailing delimiter. By default, there is no maximum length enforced. If events are malformed, this can lead to additional resource usage as events continue to be buffered in memory, and can potentially lead to memory exhaustion in extreme cases. If there is a risk of processing malformed data, such as logs with user-controlled input, consider setting the maximum length to a reasonably large value as a safety net. This ensures that processing is not actually unbounded. method required newline_delimited Byte frames which are delimited by a newline character. OctetCounting optional Byte frames according to the [octet counting](https://tools.ietf.org/html/rfc6587#section-3.4.1) format. octet_counting optional object Options for the octet counting decoder. max_length optional integer,​null The maximum length of the byte buffer. method required octet_counting Byte frames according to the [octet counting](https://tools.ietf.org/html/rfc6587#section-3.4.1) format. key required string The Redis key to read messages from. list optional Options for the Redis `list` data type. Option 1 optional object Options for the Redis `list` data type. method required Method for getting events from the `list` data type. lpop optional lpop Pop messages from the head of the list. rpop optional rpop Pop messages from the tail of the list. log_namespace optional boolean,​null The namespace to use for logs. This overrides the global setting. redis_key optional Sets the name of the log field to use to add the key to each event. The value is the Redis key that the event was read from. By default, this is not set and the field is not automatically added. Option 1 optional string An optional path that deserializes an empty string to `None`. url required string The Redis URL to connect to. The URL must take the form of `protocol://server:port/db` where the `protocol` can either be `redis` or `rediss` for connections secured using TLS. {% /tab %} {% tab title="redissource-request-example" %} ```yaml data_type: list decoding: codec: bytes framing: method: bytes key: string list: '' log_namespace: boolean redis_key: '' url: string type: redis ``` {% /tab %} ### Splunk HEC{% #splunk %} Configuration for the `splunk_hec` source. OptionsSchema {% tab title="splunk-request-model" %} FieldrequiredTypeDescription acknowledgements optional object Acknowledgement configuration for the `splunk_hec` source. ack_idle_cleanup optional boolean Whether or not to remove channels after idling for `max_idle_time` seconds. A channel is idling if it is not used for sending data or querying acknowledgement statuses. default: falseenabled optional boolean,​null Enables end-to-end acknowledgements. default: nullmax_idle_time optional integer The amount of time, in seconds, a channel is allowed to idle before removal. Channels can potentially idle for longer than this setting but clients should not rely on such behavior. Minimum of `1`. default: 300max_number_of_ack_channels optional integer The maximum number of Splunk HEC channels clients can use with this source. Minimum of `1`. default: 1000000max_pending_acks optional integer The maximum number of acknowledgement statuses pending query across all channels. Equivalent to the `max_number_of_acked_requests_pending_query` Splunk HEC setting. Minimum of `1`. default: 10000000max_pending_acks_per_channel optional integer The maximum number of acknowledgement statuses pending query for a single channel. Equivalent to the `max_number_of_acked_requests_pending_query_per_ack_channel` Splunk HEC setting. Minimum of `1`. default: 1000000address optional string The socket address to listen for connections on. The address *must* include a port. log_namespace optional boolean,​null The namespace to use for logs. This overrides the global settings. store_hec_token optional boolean Whether or not to forward the Splunk HEC authentication token with events. If set to `true`, when incoming requests contain a Splunk HEC token, the token used is kept in the event metadata and preferentially used if the event is sent to a Splunk HEC sink. tls optional Configures the TLS options for incoming/outgoing connections. Option 1 optional Configures the TLS options for incoming/outgoing connections. enabled optional boolean,​null Whether or not to require TLS for incoming or outgoing connections. When enabled and used for incoming connections, an identity certificate is also required. See `tls.crt_file` for more information. alpn_protocols optional array,​null Sets the list of supported ALPN protocols. Declare the supported ALPN protocols, which are used during negotiation with peer. They are prioritized in the order that they are defined. ca_file optional Absolute path to an additional CA certificate file. The certificate must be in the DER or PEM (X.509) format. Additionally, the certificate can be provided as an inline string in PEM format. Option 1 optional string A file path. crt_file optional Absolute path to a certificate file used to identify this server. The certificate must be in DER, PEM (X.509), or PKCS#12 format. Additionally, the certificate can be provided as an inline string in PEM format. If this is set, and is not a PKCS#12 archive, `key_file` must also be set. Option 1 optional string A file path. key_file optional Absolute path to a private key file used to identify this server. The key must be in DER or PEM (PKCS#8) format. Additionally, the key can be provided as an inline string in PEM format. Option 1 optional string A file path. key_pass optional string,​null Passphrase used to unlock the encrypted key file. This has no effect unless `key_file` is set. verify_certificate optional boolean,​null Enables certificate verification. If enabled, certificates must not be expired and must be issued by a trusted issuer. This verification operates in a hierarchical manner, checking that the leaf certificate (the certificate presented by the client/server) is not only valid, but that the issuer of that certificate is also valid, and so on until the verification process reaches a root certificate. Relevant for both incoming and outgoing connections. Do NOT set this to `false` unless you understand the risks of not verifying the validity of certificates. verify_hostname optional boolean,​null Enables hostname verification. If enabled, the hostname used to connect to the remote host must be present in the TLS certificate presented by the remote host, either as the Common Name or as an entry in the Subject Alternative Name extension. Only relevant for outgoing connections. Do NOT set this to `false` unless you understand the risks of not verifying the remote hostname. token optional Optional authorization token. **DEPRECATED**: If supplied, incoming requests must supply this token in the `Authorization` header, just as a client would if it was communicating with the Splunk HEC endpoint directly. If *not* supplied, the `Authorization` header is ignored and requests are not authenticated. Option 1 optional string Wrapper for sensitive strings containing credentials valid_tokens optional array,​null A list of valid authorization tokens. If supplied, incoming requests must supply one of these tokens in the `Authorization` header, just as a client would if it was communicating with the Splunk HEC endpoint directly. If *not* supplied, the `Authorization` header is ignored and requests are not authenticated. {% /tab %} {% tab title="splunk-request-example" %} ```yaml acknowledgements: ack_idle_cleanup: false enabled: null max_idle_time: 300 max_number_of_ack_channels: 1000000 max_pending_acks: 10000000 max_pending_acks_per_channel: 1000000 address: '0.0.0.0:8088' log_namespace: boolean store_hec_token: boolean tls: '' token: '' valid_tokens: array type: splunk_hec ``` {% /tab %} ### StatsD{% #statsd %} Configuration for the `statsd` source. OptionsSchema {% tab title="statsd-request-model" %} FieldrequiredTypeDescription Option 1 optional Listen on TCP. address required string The socket address to listen for connections on, or `systemd{#N}` to use the Nth socket passed by systemd socket activation. If a socket address is used, it *must* include a port. connection_limit optional integer,​null The maximum number of TCP connections that are allowed at any given time. keepalive optional TCP keepalive settings for socket-based components. Option 1 optional object TCP keepalive settings for socket-based components. time_secs optional integer,​null The time to wait before starting to send TCP keepalive probes on an idle connection. receive_buffer_bytes optional integer,​null The size of the receive buffer used for each connection. shutdown_timeout_secs optional integer The timeout before a connection is forcefully closed during shutdown. tls optional TlsEnableableConfig for `sources`, adding metadata from the client certificate. Option 1 optional TlsEnableableConfig for `sources`, adding metadata from the client certificate. client_metadata_key optional Event field for client certificate metadata. Option 1 optional string An optional path that deserializes an empty string to `None`. enabled optional boolean,​null Whether or not to require TLS for incoming or outgoing connections. When enabled and used for incoming connections, an identity certificate is also required. See `tls.crt_file` for more information. alpn_protocols optional array,​null Sets the list of supported ALPN protocols. Declare the supported ALPN protocols, which are used during negotiation with peer. They are prioritized in the order that they are defined. ca_file optional Absolute path to an additional CA certificate file. The certificate must be in the DER or PEM (X.509) format. Additionally, the certificate can be provided as an inline string in PEM format. Option 1 optional string A file path. crt_file optional Absolute path to a certificate file used to identify this server. The certificate must be in DER, PEM (X.509), or PKCS#12 format. Additionally, the certificate can be provided as an inline string in PEM format. If this is set, and is not a PKCS#12 archive, `key_file` must also be set. Option 1 optional string A file path. key_file optional Absolute path to a private key file used to identify this server. The key must be in DER or PEM (PKCS#8) format. Additionally, the key can be provided as an inline string in PEM format. Option 1 optional string A file path. key_pass optional string,​null Passphrase used to unlock the encrypted key file. This has no effect unless `key_file` is set. verify_certificate optional boolean,​null Enables certificate verification. If enabled, certificates must not be expired and must be issued by a trusted issuer. This verification operates in a hierarchical manner, checking that the leaf certificate (the certificate presented by the client/server) is not only valid, but that the issuer of that certificate is also valid, and so on until the verification process reaches a root certificate. Relevant for both incoming and outgoing connections. Do NOT set this to `false` unless you understand the risks of not verifying the validity of certificates. verify_hostname optional boolean,​null Enables hostname verification. If enabled, the hostname used to connect to the remote host must be present in the TLS certificate presented by the remote host, either as the Common Name or as an entry in the Subject Alternative Name extension. Only relevant for outgoing connections. Do NOT set this to `false` unless you understand the risks of not verifying the remote hostname. mode required tcp Listen on TCP. Option 2 optional Listen on UDP. address required string The socket address to listen for connections on, or `systemd{#N}` to use the Nth socket passed by systemd socket activation. If a socket address is used, it *must* include a port. receive_buffer_bytes optional integer,​null The size of the receive buffer used for each connection. mode required udp Listen on UDP. Option 3 optional Listen on a Unix domain Socket (UDS). path required string The Unix socket path. This should be an absolute path. mode required unix Listen on a Unix domain Socket (UDS). {% /tab %} {% tab title="statsd-request-example" %} ```yaml type: statsd ``` {% /tab %} ### Stdin{% #stdin %} Configuration for the `stdin` source. OptionsSchema {% tab title="stdin-request-model" %} FieldrequiredTypeDescription decoding optional Configures how events are decoded from raw bytes. Bytes optional object Uses the raw bytes as-is. codec required bytes Uses the raw bytes as-is. Json optional Decodes the raw bytes as [JSON](https://www.json.org/). json optional object JSON-specific decoding options. lossy optional boolean Determines whether or not to replace invalid UTF-8 sequences instead of failing. When true, invalid UTF-8 sequences are replaced with the [`U+FFFD REPLACEMENT CHARACTER`](https://en.wikipedia.org/wiki/Specials_\(Unicode_block\)#Replacement_character). default: truecodec required json Decodes the raw bytes as [JSON](https://www.json.org/). Protobuf optional Decodes the raw bytes as [protobuf](https://protobuf.dev/). protobuf optional object Protobuf-specific decoding options. desc_file required string Path to desc file message_type required string message type. e.g package.message codec required protobuf Decodes the raw bytes as [protobuf](https://protobuf.dev/). Syslog optional Decodes the raw bytes as a Syslog message. Decodes either as the [RFC 3164](https://www.ietf.org/rfc/rfc3164.txt)-style format ("old" style) or the [RFC 5424](https://www.ietf.org/rfc/rfc5424.txt)-style format ("new" style, includes structured data). syslog optional object Syslog-specific decoding options. lossy optional boolean Determines whether or not to replace invalid UTF-8 sequences instead of failing. When true, invalid UTF-8 sequences are replaced with the [`U+FFFD REPLACEMENT CHARACTER`](https://en.wikipedia.org/wiki/Specials_\(Unicode_block\)#Replacement_character). default: truecodec required syslog Decodes the raw bytes as a Syslog message. Decodes either as the [RFC 3164](https://www.ietf.org/rfc/rfc3164.txt)-style format ("old" style) or the [RFC 5424](https://www.ietf.org/rfc/rfc5424.txt)-style format ("new" style, includes structured data). Native optional object Decodes the raw bytes as Vector's [native Protocol Buffers format](https://github.com/vectordotdev/vector/blob/master/lib/vector-core/proto/event.proto). This codec is **[experimental](https://vector.dev/highlights/2022-03-31-native-event-codecs)**. codec required native Decodes the raw bytes as Vector's [native Protocol Buffers format](https://github.com/vectordotdev/vector/blob/master/lib/vector-core/proto/event.proto). This codec is **[experimental](https://vector.dev/highlights/2022-03-31-native-event-codecs)**. NativeJson optional Decodes the raw bytes as Vector's [native JSON format](https://github.com/vectordotdev/vector/blob/master/lib/codecs/tests/data/native_encoding/schema.cue). This codec is **[experimental](https://vector.dev/highlights/2022-03-31-native-event-codecs)**. native_json optional object Vector's native JSON-specific decoding options. lossy optional boolean Determines whether or not to replace invalid UTF-8 sequences instead of failing. When true, invalid UTF-8 sequences are replaced with the [`U+FFFD REPLACEMENT CHARACTER`](https://en.wikipedia.org/wiki/Specials_\(Unicode_block\)#Replacement_character). default: truecodec required native_json Decodes the raw bytes as Vector's [native JSON format](https://github.com/vectordotdev/vector/blob/master/lib/codecs/tests/data/native_encoding/schema.cue). This codec is **[experimental](https://vector.dev/highlights/2022-03-31-native-event-codecs)**. Gelf optional Decodes the raw bytes as a [GELF](https://docs.graylog.org/docs/gelf) message. gelf optional object GELF-specific decoding options. lossy optional boolean Determines whether or not to replace invalid UTF-8 sequences instead of failing. When true, invalid UTF-8 sequences are replaced with the [`U+FFFD REPLACEMENT CHARACTER`](https://en.wikipedia.org/wiki/Specials_\(Unicode_block\)#Replacement_character). default: truecodec required gelf Decodes the raw bytes as a [GELF](https://docs.graylog.org/docs/gelf) message. framing optional Framing configuration. Framing handles how events are separated when encoded in a raw byte form, where each event is a frame that must be prefixed, or delimited, in a way that marks where an event begins and ends within the byte stream. Option 1 optional Framing configuration. Framing handles how events are separated when encoded in a raw byte form, where each event is a frame that must be prefixed, or delimited, in a way that marks where an event begins and ends within the byte stream. Bytes optional object Byte frames are passed through as-is according to the underlying I/O boundaries (for example, split between messages or stream segments). method required bytes Byte frames are passed through as-is according to the underlying I/O boundaries (for example, split between messages or stream segments). CharacterDelimited optional Byte frames which are delimited by a chosen character. character_delimited required object Options for the character delimited decoder. delimiter required integer The character that delimits byte sequences. max_length optional integer,​null The maximum length of the byte buffer. This length does *not* include the trailing delimiter. By default, there is no maximum length enforced. If events are malformed, this can lead to additional resource usage as events continue to be buffered in memory, and can potentially lead to memory exhaustion in extreme cases. If there is a risk of processing malformed data, such as logs with user-controlled input, consider setting the maximum length to a reasonably large value as a safety net. This ensures that processing is not actually unbounded. method required character_delimited Byte frames which are delimited by a chosen character. LengthDelimited optional object Byte frames which are prefixed by an unsigned big-endian 32-bit integer indicating the length. method required length_delimited Byte frames which are prefixed by an unsigned big-endian 32-bit integer indicating the length. NewlineDelimited optional Byte frames which are delimited by a newline character. newline_delimited optional object Options for the newline delimited decoder. max_length optional integer,​null The maximum length of the byte buffer. This length does *not* include the trailing delimiter. By default, there is no maximum length enforced. If events are malformed, this can lead to additional resource usage as events continue to be buffered in memory, and can potentially lead to memory exhaustion in extreme cases. If there is a risk of processing malformed data, such as logs with user-controlled input, consider setting the maximum length to a reasonably large value as a safety net. This ensures that processing is not actually unbounded. method required newline_delimited Byte frames which are delimited by a newline character. OctetCounting optional Byte frames according to the [octet counting](https://tools.ietf.org/html/rfc6587#section-3.4.1) format. octet_counting optional object Options for the octet counting decoder. max_length optional integer,​null The maximum length of the byte buffer. method required octet_counting Byte frames according to the [octet counting](https://tools.ietf.org/html/rfc6587#section-3.4.1) format. host_key optional Overrides the name of the log field used to add the current hostname to each event. By default, the [global `log_schema.host_key` option](https://vector.dev/docs/reference/configuration/global-options/#log_schema.host_key) is used. Option 1 optional string An optional path that deserializes an empty string to `None`. log_namespace optional boolean,​null The namespace to use for logs. This overrides the global setting. max_length optional integer The maximum buffer size, in bytes, of incoming messages. Messages larger than this are truncated. {% /tab %} {% tab title="stdin-request-example" %} ```yaml decoding: codec: bytes framing: '' host_key: '' log_namespace: boolean max_length: 102400 type: stdin ``` {% /tab %} ### Syslog{% #syslog %} Configuration for the `syslog` source. OptionsSchema {% tab title="syslog-request-model" %} FieldrequiredTypeDescription host_key optional Overrides the name of the log field used to add the peer host to each event. If using TCP or UDP, the value is the peer host's address, including the port. For example, `1.2.3.4:9000`. If using UDS, the value is the socket path itself. By default, the [global `log_schema.host_key` option](https://vector.dev/docs/reference/configuration/global-options/#log_schema.host_key) is used. Option 1 optional string An optional path that deserializes an empty string to `None`. log_namespace optional boolean,​null The namespace to use for logs. This overrides the global setting. max_length optional integer The maximum buffer size of incoming messages, in bytes. Messages larger than this are truncated. _metadata optional description optional Listener mode for the `syslog` source. oneOf optional {% /tab %} {% tab title="syslog-request-example" %} ```yaml host_key: '' log_namespace: boolean max_length: 102400 type: syslog ``` {% /tab %} ### Vector{% #vector %} Configuration for the `vector` source. OptionsSchema {% tab title="vector-request-model" %} FieldrequiredTypeDescription acknowledgements optional object Controls how acknowledgements are handled by this source. **DEPRECATED**: This setting is **deprecated** in favor of enabling `acknowledgements` at the [global](https://vector.dev/docs/reference/configuration/global-options/#acknowledgements) or sink level. Enabling or disabling acknowledgements at the source level has **no effect** on acknowledgement behavior. See [End-to-end Acknowledgements](https://vector.dev/docs/about/under-the-hood/architecture/end-to-end-acknowledgements/) for more information on how event acknowledgement is handled. enabled optional boolean,​null Whether or not end-to-end acknowledgements are enabled for this source. default: nulladdress required string The socket address to listen for connections on. It *must* include a port. log_namespace optional boolean,​null The namespace to use for logs. This overrides the global setting. tls optional Configures the TLS options for incoming/outgoing connections. Option 1 optional Configures the TLS options for incoming/outgoing connections. enabled optional boolean,​null Whether or not to require TLS for incoming or outgoing connections. When enabled and used for incoming connections, an identity certificate is also required. See `tls.crt_file` for more information. alpn_protocols optional array,​null Sets the list of supported ALPN protocols. Declare the supported ALPN protocols, which are used during negotiation with peer. They are prioritized in the order that they are defined. ca_file optional Absolute path to an additional CA certificate file. The certificate must be in the DER or PEM (X.509) format. Additionally, the certificate can be provided as an inline string in PEM format. Option 1 optional string A file path. crt_file optional Absolute path to a certificate file used to identify this server. The certificate must be in DER, PEM (X.509), or PKCS#12 format. Additionally, the certificate can be provided as an inline string in PEM format. If this is set, and is not a PKCS#12 archive, `key_file` must also be set. Option 1 optional string A file path. key_file optional Absolute path to a private key file used to identify this server. The key must be in DER or PEM (PKCS#8) format. Additionally, the key can be provided as an inline string in PEM format. Option 1 optional string A file path. key_pass optional string,​null Passphrase used to unlock the encrypted key file. This has no effect unless `key_file` is set. verify_certificate optional boolean,​null Enables certificate verification. If enabled, certificates must not be expired and must be issued by a trusted issuer. This verification operates in a hierarchical manner, checking that the leaf certificate (the certificate presented by the client/server) is not only valid, but that the issuer of that certificate is also valid, and so on until the verification process reaches a root certificate. Relevant for both incoming and outgoing connections. Do NOT set this to `false` unless you understand the risks of not verifying the validity of certificates. verify_hostname optional boolean,​null Enables hostname verification. If enabled, the hostname used to connect to the remote host must be present in the TLS certificate presented by the remote host, either as the Common Name or as an entry in the Subject Alternative Name extension. Only relevant for outgoing connections. Do NOT set this to `false` unless you understand the risks of not verifying the remote hostname. version optional Version of the configuration. Option 1 optional Marker type for version two of the configuration for the `vector` source. 2 optional 2 Marker value for version two. {% /tab %} {% tab title="vector-request-example" %} ```yaml acknowledgements: enabled: null address: string log_namespace: boolean tls: '' version: '' type: vector ``` {% /tab %}