Este producto no es compatible con el sitio Datadog seleccionado. ().
Esta página aún no está disponible en español. Estamos trabajando en su traducción. Si tienes alguna pregunta o comentario sobre nuestro actual proyecto de traducción, no dudes en ponerte en contacto con nosotros.
Disponible para:
Logs|Metrics|Traces
Overview
Use this processor with Vector Remap Language (VRL) to modify and enrich your logs, metrics, or Contact your account manager to request access.. VRL is an expression-oriented, domain specific language designed for transforming data. It features built-in functions for observability use cases. You can use custom functions in the following ways:
See Remap Reserved Attributes on how to use the Custom Processor to manually and dynamically remap attributes.
Setup
To set up this processor:
If you have not created any functions yet, click Add custom processor and follow the instructions in Add a function to create a function.
If you have already added custom functions, click Manage custom processors. Click on a function in the list to edit or delete it. You can use the search bar to find a function by its name. Click Add Custom Processor to add a function.
Add a function
Enter a name for your custom processor.
Add your script to modify your data using custom functions. You can also click Autofill with Example and select one of the common use cases to get started. Click the copy icon for the example script and paste it into your script. See Get Started with the Custom Processor for more information.
Optionally, check Drop events on error if you want to drop events that encounter an error during processing.
Enter a sample event.
Click Run to preview how the functions process the event. After the script has run, you can see the output for the event.
Click Save.
Custom functions
The functions are organized into the following categories:
Appends each item in the items array to the end of the value array.
argument
Tipo
Descripción
default
required
value
array
The initial array.
N/A
yes
items
array
The items to append.
N/A
yes
Examples
Append to an array
Source:
append([1, 2], [3, 4])
Return:
[1,2,3,4]
chunks
Chunks value into slices of length chunk_size bytes.
argument
Tipo
Descripción
default
required
value
array, string
The array of bytes to split.
N/A
yes
chunk_size
integer
The desired length of each chunk in bytes. This may be constrained by the host platform architecture.
N/A
yes
Errors
chunk_size must be at least 1 byte.
chunk_size is too large.
Examples
Split a string into chunks
Source:
chunks("abcdefgh", 4)
Return:
["abcd","efgh"]
Chunks do not respect unicode code point boundaries
Source:
chunks("ab你好", 4)
Return:
["ab�","�好"]
pop
Removes the last item from the value array, returning a new array without the last element.
argument
Tipo
Descripción
default
required
value
array
The array to pop from.
N/A
yes
Examples
Pop an item from an array
Source:
pop([1, 2, 3])
Return:
[1,2]
push
Adds the item to the end of the value array.
argument
Tipo
Descripción
default
required
value
array
The target array.
N/A
yes
item
any
The item to push.
N/A
yes
Examples
Push an item onto an array
Source:
push([1, 2], 3)
Return:
[1,2,3]
zip
Iterate over several arrays in parallel, producing a new array containing arrays of items from each source.
The resulting array will be as long as the shortest input array, with all the remaining elements dropped.
This function is modeled from the zip function in Python,
but similar methods can be found in Ruby
and Rust.
If a single parameter is given, it must contain an array of all the input arrays.
argument
Tipo
Descripción
default
required
array_0
array
The first array of elements, or the array of input arrays if no other parameter is present.
N/A
yes
array_1
array
The second array of elements. If not present, the first parameter contains all the arrays.
N/A
no
Errors
array_0 and array_1 must be arrays.
Examples
Merge two arrays
Source:
zip([1, 2, 3], [4, 5, 6, 7])
Return:
[[1,4],[2,5],[3,6]]
Merge three arrays
Source:
zip([[1, 2], [3, 4], [5, 6]])
Return:
[[1,3,5],[2,4,6]]
Checksum Functions
crc
Calculates a CRC of the value.
The CRC algorithm used can be optionally specified.
This function is infallible if either the default algorithm value or a recognized-valid compile-time
algorithm string literal is used. Otherwise, it is fallible.
argument
Tipo
Descripción
default
required
value
string
The string to calculate the checksum for.
N/A
yes
algorithm
string
The CRC algorithm to use.
CRC_32_ISO_HDLC
no
Errors
value is not a string.
algorithm is not a supported algorithm.
Examples
Create CRC checksum using the default algorithm
Source:
crc("foo")
Return:
"2356372769"
Create CRC checksum using the CRC_32_CKSUM algorithm
Source:
crc("foo", algorithm: "CRC_32_CKSUM")
Return:
"4271552933"
Codec Functions
decode_base16
Decodes the value (a Base16 string) into its original string.
Decodes a punycode encoded value, such as an internationalized domain name (IDN). This function assumes that the value passed is meant to be used in IDN context and that it is either a domain name or a part of it.
argument
Tipo
Descripción
default
required
value
string
The string to decode.
N/A
yes
validate
boolean
If enabled, checks if the input string is a valid domain name.
true
no
Errors
value is not valid punycode
Examples
Decode a punycode encoded internationalized domain name
Encodes a value with percent encoding to safely be used in URLs.
argument
Tipo
Descripción
default
required
value
string
The string to encode.
N/A
yes
ascii_set
string
The ASCII set to use when encoding the data.
NON_ALPHANUMERIC
no
Examples
Percent encode all non-alphanumeric characters (default)
Source:
encode_percent("foo bar?")
Return:
"foo%20bar%3F"
Percent encode only control characters
Source:
encode_percent("foo bar", ascii_set: "CONTROLS")
Return:
"foo %09bar"
encode_proto
Encodes the value into a protocol buffer payload.
argument
Tipo
Descripción
default
required
value
object
The object to convert to a protocol buffer payload.
N/A
yes
desc_file
string
The path to the protobuf descriptor set file. Must be a literal string.
This file is the output of protoc -o …
N/A
yes
message_type
string
The name of the message type to use for serializing.
Must be a literal string.
N/A
yes
allow_lossy_string_coercion
boolean
Whether to permit lossy coercion of Boolean, Integer, Float, and Timestamp values into protobuf string fields by stringifying the value. Defaults to true to preserve permissive behavior; set to false for strict, spec-compliant encoding (see the protobuf JSON mapping).
true
no
Errors
desc_file file does not exist.
message_type message type does not exist in the descriptor file.
Encodes a value to punycode. Useful for internationalized domain names (IDN). This function assumes that the value passed is meant to be used in IDN context and that it is either a domain name or a part of it.
argument
Tipo
Descripción
default
required
value
string
The string to encode.
N/A
yes
validate
boolean
Whether to validate the input string to check if it is a valid domain name.
true
no
Errors
value can not be encoded to punycode
Examples
Encode an internationalized domain name
Source:
encode_punycode!("www.café.com")
Return:
"www.xn--caf-dma.com"
Encode an internationalized domain name with mixed case
The value to convert to a float. Must be convertible to a float, otherwise an error is raised.
N/A
yes
Errors
value is not a supported float representation.
Examples
Coerce to a float
Source:
to_float!("3.145")
Return:
3.145
Coerce to a float (timestamp)
Source:
to_float(t'2020-12-30T22:20:53.824727Z')
Return:
1609366853.824727
to_int
Coerces the value into an integer.
argument
Tipo
Descripción
default
required
value
integer, float, boolean, string, timestamp, null
The value to convert to an integer.
N/A
yes
Errors
value is a string but the text is not an integer.
value is not a string, int, or timestamp.
Examples
Coerce to an int (string)
Source:
to_int!("2")
Return:
2
Coerce to an int (timestamp)
Source:
to_int(t'2020-12-30T22:20:53.824727Z')
Return:
1609366853
to_regex
Coerces the value into a regex.
argument
Tipo
Descripción
default
required
value
string
The value to convert to a regex.
N/A
yes
Errors
value is not a string.
Examples
Coerce to a regex
Source:
to_regex("^foo$") ?? r''
Return:
"^foo$"
to_string
Coerces the value into a string.
argument
Tipo
Descripción
default
required
value
integer, float, boolean, string, timestamp, null
The value to convert to a string.
N/A
yes
Errors
value is not an integer, float, boolean, string, timestamp, or null.
Examples
Coerce to a string (Boolean)
Source:
to_string(true)
Return:
"true"
Coerce to a string (int)
Source:
to_string(52)
Return:
"52"
Coerce to a string (float)
Source:
to_string(52.2)
Return:
"52.2"
Convert Functions
from_unix_timestamp
Converts the value integer from a Unix timestamp to a DPL timestamp.
Converts from the number of seconds since the Unix epoch by default. To convert from milliseconds or nanoseconds, set the unit argument to milliseconds or nanoseconds.
argument
Tipo
Descripción
default
required
value
integer
The Unix timestamp to convert.
N/A
yes
unit
string
The time unit.
seconds
no
Examples
Convert from a Unix timestamp (seconds)
Source:
from_unix_timestamp!(5)
Return:
"1970-01-01T00:00:05Z"
Convert from a Unix timestamp (milliseconds)
Source:
from_unix_timestamp!(5000, unit: "milliseconds")
Return:
"1970-01-01T00:00:05Z"
Convert from a Unix timestamp (nanoseconds)
Source:
from_unix_timestamp!(5000, unit: "nanoseconds")
Return:
"1970-01-01T00:00:00.000005Z"
to_syslog_facility
Converts the value, a Syslog facility code, into its corresponding
Syslog keyword. For example, 0 into "kern", 1 into "user", etc.
Returns the number of seconds since the Unix epoch by default. To return the number in milliseconds or nanoseconds, set the unit argument to milliseconds or nanoseconds.
AES-256-CBC-ANSIX923 (key = 32 bytes, iv = 16 bytes)
AES-192-CBC-ANSIX923 (key = 24 bytes, iv = 16 bytes)
AES-128-CBC-ANSIX923 (key = 16 bytes, iv = 16 bytes)
AES-256-CBC-ISO7816 (key = 32 bytes, iv = 16 bytes)
AES-192-CBC-ISO7816 (key = 24 bytes, iv = 16 bytes)
AES-128-CBC-ISO7816 (key = 16 bytes, iv = 16 bytes)
AES-256-CBC-ISO10126 (key = 32 bytes, iv = 16 bytes)
AES-192-CBC-ISO10126 (key = 24 bytes, iv = 16 bytes)
AES-128-CBC-ISO10126 (key = 16 bytes, iv = 16 bytes)
CHACHA20-POLY1305 (key = 32 bytes, iv = 12 bytes)
XCHACHA20-POLY1305 (key = 32 bytes, iv = 24 bytes)
XSALSA20-POLY1305 (key = 32 bytes, iv = 24 bytes)
argument
Tipo
Descripción
default
required
ciphertext
string
The string in raw bytes (not encoded) to decrypt.
N/A
yes
algorithm
string
The algorithm to use.
N/A
yes
key
string
The key in raw bytes (not encoded) for decryption. The length must match the algorithm requested.
N/A
yes
iv
string
The IV in raw bytes (not encoded) for decryption. The length must match the algorithm requested.
A new IV should be generated for every message. You can use random_bytes to generate a cryptographically secure random value.
The value should match the one used during encryption.
N/A
yes
Errors
algorithm is not a supported algorithm.
key length does not match the key size required for the algorithm specified.
iv length does not match the iv size required for the algorithm specified.
AES-256-CBC-ANSIX923 (key = 32 bytes, iv = 16 bytes)
AES-192-CBC-ANSIX923 (key = 24 bytes, iv = 16 bytes)
AES-128-CBC-ANSIX923 (key = 16 bytes, iv = 16 bytes)
AES-256-CBC-ISO7816 (key = 32 bytes, iv = 16 bytes)
AES-192-CBC-ISO7816 (key = 24 bytes, iv = 16 bytes)
AES-128-CBC-ISO7816 (key = 16 bytes, iv = 16 bytes)
AES-256-CBC-ISO10126 (key = 32 bytes, iv = 16 bytes)
AES-192-CBC-ISO10126 (key = 24 bytes, iv = 16 bytes)
AES-128-CBC-ISO10126 (key = 16 bytes, iv = 16 bytes)
CHACHA20-POLY1305 (key = 32 bytes, iv = 12 bytes)
XCHACHA20-POLY1305 (key = 32 bytes, iv = 24 bytes)
XSALSA20-POLY1305 (key = 32 bytes, iv = 24 bytes)
argument
Tipo
Descripción
default
required
plaintext
string
The string to encrypt.
N/A
yes
algorithm
string
The algorithm to use.
N/A
yes
key
string
The key in raw bytes (not encoded) for encryption. The length must match the algorithm requested.
N/A
yes
iv
string
The IV in raw bytes (not encoded) for encryption. The length must match the algorithm requested.
A new IV should be generated for every message. You can use random_bytes to generate a cryptographically secure random value.
N/A
yes
Errors
algorithm is not a supported algorithm.
key length does not match the key size required for the algorithm specified.
iv length does not match the iv size required for the algorithm specified.
Examples
Encrypt value
Source:
plaintext="super secret message"iv="1234567890123456"# typically you would call random_bytes(16)key="16_byte_keyxxxxx"encrypted_message= encrypt!(plaintext, "AES-128-CBC-PKCS7", key, iv: iv)encode_base64(encrypted_message)
Return:
"GBw8Mu00v0Kc38+/PvsVtGgWuUJ+ZNLgF8Opy8ohIYE="
hmac
Calculates a HMAC of the value using the given key.
The hashing algorithm used can be optionally specified.
For most use cases, the resulting bytestream should be encoded into a hex or base64
string using either encode_base16 or
encode_base64.
This function is infallible if either the default algorithm value or a recognized-valid compile-time
algorithm string literal is used. Otherwise, it is fallible.
argument
Tipo
Descripción
default
required
value
string
The string to calculate the HMAC for.
N/A
yes
key
string
The string to use as the cryptographic key.
N/A
yes
algorithm
string
The hashing algorithm to use.
SHA-256
no
Examples
Calculate message HMAC (defaults: SHA-256), encoding to a base64 string
Calculates a Seahash hash of the value.
Note: Due to limitations in the underlying DPL data types, this function converts the unsigned 64-bit integer SeaHash result to a signed 64-bit integer. Results higher than the signed 64-bit integer maximum value wrap around to negative values.
Asserts the condition, which must be a Boolean expression. The program is aborted with
message if the condition evaluates to false.
argument
Tipo
Descripción
default
required
condition
boolean
The condition to check.
N/A
yes
message
string
An optional custom error message. If the equality assertion fails, message is
appended to the default message prefix. See the examples below
for a fully formed log message sample.
N/A
no
Errors
condition evaluates to false.
Examples
Assertion (true)
Source:
assert!("foo"=="foo", message: "\"foo\" must be \"foo\"!")
Return:
true
Assertion (false)
Source:
assert!("foo"=="bar", message: "\"foo\" must be \"foo\"!")
assert_eq
Asserts that two expressions, left and right, have the same value. The program is
aborted with message if they do not have the same value.
argument
Tipo
Descripción
default
required
left
any
The value to check for equality against right.
N/A
yes
right
any
The value to check for equality against left.
N/A
yes
message
string
An optional custom error message. If the equality assertion fails, message is
appended to the default message prefix. See the examples
below for a fully formed log message sample.
Searches an enrichment table for rows that match the
provided condition.
For file enrichment tables, this condition needs to be a DPL object in which
the key-value pairs indicate a field to search mapped to a value to search in that field.
This function returns the rows that match the provided condition(s). All fields need to
match for rows to be returned; if any fields do not match, then no rows are returned.
There are currently two forms of search criteria:
Exact match search. The given field must match the value exactly. Case sensitivity
can be specified using the case_sensitive argument. An exact match search can use an
index directly into the dataset, which should make this search fairly “cheap” from a
performance perspective.
Date range search. The given field must be greater than or equal to the from date
and/or less than or equal to the to date. A date range search involves
sequentially scanning through the rows that have been located using any exact match
criteria. This can be an expensive operation if there are many rows returned by any exact
match criteria. Therefore, use date ranges as the only criteria when the enrichment
data set is very small.
For geoip and mmdb enrichment tables, this condition needs to be a DPL object with a single key-value pair
whose value needs to be a valid IP address. Example: {"ip": .ip }. If a return field is expected
and without a value, null is used. This table can return the following fields:
ISP databases:
autonomous_system_number
autonomous_system_organization
isp
organization
City databases:
city_name
continent_code
country_code
country_name
region_code
region_name
metro_code
latitude
longitude
postal_code
timezone
Connection-Type databases:
connection_type
To use this function, you need to update your configuration to
include an
enrichment_tables
parameter.
argument
Tipo
Descripción
default
required
table
string
The enrichment table to search.
N/A
yes
condition
object
The condition to search on. Since the condition is used at boot time to create
indices into the data, these conditions must be statically defined.
N/A
yes
select
array
A subset of fields from the enrichment table to return. If not specified,
all fields are returned.
Searches an enrichment table for a row that matches the
provided condition. A single row must be matched. If no rows are found or more than one row is
found, an error is returned.
For file enrichment tables, this condition needs to be a DPL object in which
the key-value pairs indicate a field to search mapped to a value to search in that field.
This function returns the rows that match the provided condition(s). All fields need to
match for rows to be returned; if any fields do not match, then no rows are returned.
There are currently two forms of search criteria:
Exact match search. The given field must match the value exactly. Case sensitivity
can be specified using the case_sensitive argument. An exact match search can use an
index directly into the dataset, which should make this search fairly “cheap” from a
performance perspective.
Date range search. The given field must be greater than or equal to the from date
and/or less than or equal to the to date. A date range search involves
sequentially scanning through the rows that have been located using any exact match
criteria. This can be an expensive operation if there are many rows returned by any exact
match criteria. Therefore, use date ranges as the only criteria when the enrichment
data set is very small.
For geoip and mmdb enrichment tables, this condition needs to be a DPL object with a single key-value pair
whose value needs to be a valid IP address. Example: {"ip": .ip }. If a return field is expected
and without a value, null is used. This table can return the following fields:
ISP databases:
autonomous_system_number
autonomous_system_organization
isp
organization
City databases:
city_name
continent_code
country_code
country_name
region_code
region_name
metro_code
latitude
longitude
postal_code
timezone
Connection-Type databases:
connection_type
To use this function, you need to update your configuration to
include an
enrichment_tables
parameter.
argument
Tipo
Descripción
default
required
table
string
The enrichment table to search.
N/A
yes
condition
object
The condition to search on. Since the condition is used at boot time to create
indices into the data, these conditions must be statically defined.
N/A
yes
select
array
A subset of fields from the enrichment table to return. If not specified,
all fields are returned.
This function currently does not support recursive iteration.
The function uses the function closure syntax to allow reading
the key-value or index-value combination for each item in the
collection.
The same scoping rules apply to closure blocks as they do for
regular blocks. This means that any variable defined in parent scopes
is accessible, and mutations to those variables are preserved,
but any new variables instantiated in the closure block are
unavailable outside of the block.
See the examples below to learn about the closure syntax.
argument
Tipo
Descripción
default
required
value
array, object
The array or object to filter.
N/A
yes
Examples
Filter elements
Source:
filter(array!(.tags)) -> |_index, value|{# keep any elements that aren't equal to "foo" value !="foo"}
Return:
["bar","baz"]
flatten
Flattens the value into a single-level representation.
This function currently does not support recursive iteration.
The function uses the “function closure syntax” to allow reading
the key/value or index/value combination for each item in the
collection.
The same scoping rules apply to closure blocks as they do for
regular blocks. This means that any variable defined in parent scopes
is accessible, and mutations to those variables are preserved,
but any new variables instantiated in the closure block are
unavailable outside of the block.
See the examples below to learn about the closure syntax.
argument
Tipo
Descripción
default
required
value
array, object
The array or object to iterate.
N/A
yes
Examples
Tally elements
Source:
tally={}for_each(array!(.tags)) -> |_index, value|{# Get the current tally for the `value`, or# set to `0`.count= int(get!(tally, [value])) ?? 0# Increment the tally for the value by `1`.tally= set!(tally, [value], count + 1)}tally
Return:
{"foo":2,"bar":1,"baz":1}
includes
Determines whether the value array includes the specified item.
argument
Tipo
Descripción
default
required
value
array
The array.
N/A
yes
item
any
The item to check.
N/A
yes
Examples
Array includes
Source:
includes(["apple", "orange", "banana"], "banana")
Return:
true
keys
Returns the keys from the object passed into the function.
argument
Tipo
Descripción
default
required
value
object
The object to extract keys from.
N/A
yes
Examples
Get keys from the object
Source:
keys({"key1": "val1", "key2": "val2"})
Return:
["key1","key2"]
length
Returns the length of the value.
If value is an array, returns the number of elements.
If value is an object, returns the number of top-level keys.
If value is a string, returns the number of bytes in the string. If
you want the number of characters, see strlen.
If recursive is enabled, the function iterates into nested
objects, using the following rules:
Iteration starts at the root.
For every nested object type:
First return the key of the object type itself.
Then recurse into the object, and loop back to item (1)
in this list.
Any mutation done on a nested object before recursing into
it, are preserved.
For every nested array type:
First return the key of the array type itself.
Then find all objects within the array, and apply item (2)
to each individual object.
The above rules mean that map_keys with
recursive enabled finds all keys in the target,
regardless of whether nested objects are nested inside arrays.
The function uses the function closure syntax to allow reading
the key for each item in the object.
The same scoping rules apply to closure blocks as they do for
regular blocks. This means that any variable defined in parent scopes
is accessible, and mutations to those variables are preserved,
but any new variables instantiated in the closure block are
unavailable outside of the block.
See the examples below to learn about the closure syntax.
If recursive is enabled, the function iterates into nested
collections, using the following rules:
Iteration starts at the root.
For every nested collection type:
First return the collection type itself.
Then recurse into the collection, and loop back to item (1)
in the list
Any mutation done on a collection before recursing into it,
are preserved.
The function uses the function closure syntax to allow mutating
the value for each item in the collection.
The same scoping rules apply to closure blocks as they do for
regular blocks, meaning, any variable defined in parent scopes
are accessible, and mutations to those variables are preserved,
but any new variables instantiated in the closure block are
unavailable outside of the block.
Check out the examples below to learn about the closure syntax.
argument
Tipo
Descripción
default
required
value
array, object
The object or array to iterate.
N/A
yes
recursive
boolean
Whether to recursively iterate the collection.
false
no
Examples
Upcase values
Source:
map_values(.) -> |value|{ upcase!(value)}
Return:
{"foo":"FOO","bar":"BAR"}
match_array
Determines whether the elements in the value array matches the pattern. By default, it checks that at least one element matches, but can be set to determine if all the elements match.
Encrypts an IP address, transforming it into a different valid IP address of the same version.
Supported modes:
aes128 - Scrambles the IP using AES-128 encryption (ipcrypt-deterministic). Accepts IPv4 or IPv6; key must be exactly 16 bytes.
pfx - Prefix-preserving encryption (ipcrypt-pfx), that maintains network hierarchy so addresses in the same subnet encrypt to addresses sharing the same prefix. Key must be exactly 32 bytes.
Encryption is deterministic: the same input IP, key, and mode always produce the same output.
argument
Tipo
Descripción
default
required
ip
string
The IP address to encrypt (IPv4 or IPv6).
N/A
yes
key
string
The encryption key as raw bytes. Must be 16 bytes for aes128 mode or 32 bytes for pfx mode.
N/A
yes
mode
string
The encryption mode: aes128 or pfx.
N/A
yes
Errors
ip is not a valid IP address.
mode is not a supported mode (aes128 or pfx).
key length does not match the mode requirements (16 bytes for aes128, 32 bytes for pfx).
Extracts the subnet address from the ip using the supplied subnet.
argument
Tipo
Descripción
default
required
ip
string
The IP address (v4 or v6).
N/A
yes
subnet
string
The subnet to extract from the IP address. This can be either a prefix length like /8 or a net mask
like 255.255.0.0. The net mask can be either an IPv4 or IPv6 address.
N/A
yes
Errors
ip is not a valid IP address.
subnet is not a valid subnet.
Examples
IPv4 subnet
Source:
ip_subnet!("192.168.10.32", "255.255.255.0")
Return:
"192.168.10.0"
IPv6 subnet
Source:
ip_subnet!("2404:6800:4003:c02::64", "/32")
Return:
"2404:6800::"
ip_to_ipv6
Converts the ip to an IPv6 address.
argument
Tipo
Descripción
default
required
ip
string
The IP address to convert to IPv6.
N/A
yes
Errors
ip is not a valid IP address.
Examples
IPv4 to IPv6
Source:
ip_to_ipv6!("192.168.10.32")
Return:
"::ffff:192.168.10.32"
ipv6_to_ipv4
Converts the ip to an IPv4 address. ip is returned unchanged if it’s already an IPv4 address. If ip is
currently an IPv6 address then it needs to be IPv4 compatible, otherwise an error is thrown.
argument
Tipo
Descripción
default
required
ip
string
The IPv4-mapped IPv6 address to convert.
N/A
yes
Errors
ip is not a valid IP address.
ip is an IPv6 address that is not compatible with IPv4.
Examples
IPv6 to IPv4
Source:
ipv6_to_ipv4!("::ffff:192.168.0.1")
Return:
"192.168.0.1"
is_ipv4
Check if the string is a valid IPv4 address or not.
An IPv4-mapped or
IPv4-compatible IPv6 address is not considered
valid for the purpose of this function.
Iterate over either one array of arrays or a pair of arrays and create an object out of all the key-value pairs contained in them.
With one array of arrays, any entries with no value use null instead.
Any keys that are null skip the corresponding value.
If a single parameter is given, it must contain an array of all the input arrays.
argument
Tipo
Descripción
default
required
values
array
The first array of elements, or the array of input arrays if no other parameter is present.
N/A
yes
keys
array
The second array of elements. If not present, the first parameter must contain all the arrays.
N/A
no
Errors
values and keys must be arrays.
If keys is not present, values must contain only arrays.
parse_apache_log!( s'127.0.0.1 bob frank [10/Oct/2000:13:55:36 -0700] "GET /apache_pb.gif HTTP/1.0" 200 2326 "http://www.seniorinfomediaries.com/vertical/channels/front-end/bandwidth" "Mozilla/5.0 (X11; Linux i686; rv:5.0) Gecko/1945-10-12 Firefox/37.0"',
"combined",
)
Return:
{"host":"127.0.0.1","identity":"bob","user":"frank","timestamp":"2000-10-10T20:55:36Z","message":"GET /apache_pb.gif HTTP/1.0","method":"GET","path":"/apache_pb.gif","protocol":"HTTP/1.0","status":200,"size":2326,"referrer":"http://www.seniorinfomediaries.com/vertical/channels/front-end/bandwidth","agent":"Mozilla/5.0 (X11; Linux i686; rv:5.0) Gecko/1945-10-12 Firefox/37.0"}
Parse using Apache log format (error)
Source:
parse_apache_log!( s'[01/Mar/2021:12:00:19 +0000] [ab:alert] [pid 4803:tid 3814] [client 147.159.108.175:24259] I will bypass the haptic COM bandwidth, that should matrix the CSS driver!',
"error")
Return:
{"client":"147.159.108.175","message":"I will bypass the haptic COM bandwidth, that should matrix the CSS driver!","module":"ab","pid":4803,"port":24259,"severity":"alert","thread":"3814","timestamp":"2021-03-01T12:00:19Z"}
Parses the value in CEF (Common Event Format) format. Ignores everything up to CEF header. Empty values are returned as empty strings. Surrounding quotes are removed from values.
argument
Tipo
Descripción
default
required
value
string
The string to parse.
N/A
yes
translate_custom_fields
boolean
Toggles translation of custom field pairs to key:value.
{"dataType":"Message","dataTypeId":1,"extraInfo":"","messageType":"ResolverQuery","messageTypeId":3,"queryZone":"com.","requestData":{"fullRcode":0,"header":{"aa":false,"ad":false,"anCount":0,"arCount":1,"cd":false,"id":37634,"nsCount":0,"opcode":0,"qdCount":1,"qr":0,"ra":false,"rcode":0,"rd":false,"tc":false},"opt":{"do":true,"ednsVersion":0,"extendedRcode":0,"options":[{"optCode":10,"optName":"Cookie","optValue":"7GMIAb3NWDM="}],"udpPayloadSize":512},"question":[{"class":"IN","domainName":"facebook1.com.","questionType":"A","questionTypeId":1}],"rcodeName":"NoError"},"responseData":{"fullRcode":16,"header":{"aa":false,"ad":false,"anCount":0,"arCount":1,"cd":false,"id":45880,"nsCount":0,"opcode":0,"qdCount":1,"qr":0,"ra":false,"rcode":16,"rd":false,"tc":false},"opt":{"do":false,"ednsVersion":1,"extendedRcode":1,"ede":[{"extraText":"no SEP matching the DS found for dnssec-failed.org.","infoCode":9,"purpose":"DNSKEY Missing"}],"udpPayloadSize":1232},"question":[{"class":"IN","domainName":"h5.example.com.","questionType":"SOA","questionTypeId":6}],"rcodeName":"BADSIG"},"responseAddress":"2001:502:7094::30","responsePort":53,"serverId":"james-Virtual-Machine","serverVersion":"BIND 9.16.3","socketFamily":"INET6","socketProtocol":"UDP","sourceAddress":"::","sourcePort":46835,"time":1593489007920014000,"timePrecision":"ns","timestamp":"2020-06-30T03:50:07.920014129Z"}
parse_duration
Parses the value into a human-readable duration format specified by unit.
argument
Tipo
Descripción
default
required
value
string
The string of the duration.
N/A
yes
unit
string
The output units for the duration.
N/A
yes
Errors
value is not a properly formatted duration.
Examples
Parse duration (milliseconds)
Source:
parse_duration!("1005ms", unit: "s")
Return:
1.005
Parse multiple durations (seconds & milliseconds)
Source:
parse_duration!("1s 1ms", unit: "ms")
Return:
1001
parse_etld
Parses the eTLD from value representing domain name.
argument
Tipo
Descripción
default
required
value
string
The domain string.
N/A
yes
plus_parts
integer
Can be provided to get additional parts of the domain name. When 1 is passed,
eTLD+1 will be returned, which represents a domain registrable by a single
organization. Higher numbers will return subdomains.
false
no
psl
string
Can be provided to use a different public suffix list.
Parses the string value representing a number in an optional base/radix to an integer.
argument
Tipo
Descripción
default
required
value
string
The string to parse.
N/A
yes
base
integer
The base the number is in. Must be between 2 and 36 (inclusive).
If unspecified, the string prefix is used to
determine the base: “0b”, 8 for “0” or “0o”, 16 for “0x”,
and 10 otherwise.
N/A
no
Errors
The base is not between 2 and 36.
The number cannot be parsed in the base.
Examples
Parse decimal
Source:
parse_int!("-42")
Return:
-42
Parse binary
Source:
parse_int!("0b1001")
Return:
9
Parse octal
Source:
parse_int!("0o42")
Return:
34
Parse hexadecimal
Source:
parse_int!("0x2a")
Return:
42
Parse explicit base
Source:
parse_int!("2a", 17)
Return:
44
parse_json
Parses the value as JSON.
argument
Tipo
Descripción
default
required
value
string
The string representation of the JSON to parse.
N/A
yes
max_depth
integer
Number of layers to parse for nested JSON-formatted documents.
The value must be in the range of 1 to 128.
N/A
no
lossy
boolean
Whether to parse the JSON in a lossy manner. Replaces invalid UTF-8 characters
with the Unicode character � (U+FFFD) if set to true, otherwise returns an error
if there are any invalid UTF-8 characters present.
Parses the value in key-value format. Also known as logfmt.
Keys and values can be wrapped with ".
" characters can be escaped using \.
argument
Tipo
Descripción
default
required
value
string
The string to parse.
N/A
yes
key_value_delimiter
string
The string that separates the key from the value.
=
no
field_delimiter
string
The string that separates each key-value pair.
N/A
no
whitespace
string
Defines the acceptance of unnecessary whitespace surrounding the configured key_value_delimiter.
lenient
no
accept_standalone_key
boolean
Whether a standalone key should be accepted, the resulting object associates such keys with the boolean value true.
true
no
Errors
value is not a properly formatted key-value string.
Examples
Parse logfmt log
Source:
parse_key_value!("@timestamp=\"Sun Jan 10 16:47:39 EST 2021\" level=info msg=\"Stopping all fetchers\" tag#production=stopping_fetchers id=ConsumerFetcherManager-1382721708341 module=kafka.consumer.ConsumerFetcherManager")
Return:
{"@timestamp":"Sun Jan 10 16:47:39 EST 2021","level":"info","msg":"Stopping all fetchers","tag#production":"stopping_fetchers","id":"ConsumerFetcherManager-1382721708341","module":"kafka.consumer.ConsumerFetcherManager"}
Parses the value using the klog format used by Kubernetes components.
argument
Tipo
Descripción
default
required
value
string
The string to parse.
N/A
yes
Errors
value does not match the klog format.
Examples
Parse using klog
Source:
parse_klog!("I0505 17:59:40.692994 28133 klog.go:70] hello from klog")
Return:
{"file":"klog.go","id":28133,"level":"info","line":70,"message":"hello from klog","timestamp":"2025-05-05T17:59:40.692994Z"}
parse_linux_authorization
Parses Linux authorization logs usually found under either /var/log/auth.log (for Debian-based systems) or
/var/log/secure (for RedHat-based systems) according to Syslog format.
argument
Tipo
Descripción
default
required
value
string
The text containing the message to parse.
N/A
yes
Errors
value is not a properly formatted Syslog message.
Examples
Parse Linux authorization event
Source:
parse_linux_authorization!( s'Mar 23 01:49:58 localhost sshd[1111]: Accepted publickey for eng from 10.1.1.1 port 8888 ssh2: RSA SHA256:foobar')
Return:
{"appname":"sshd","hostname":"localhost","message":"Accepted publickey for eng from 10.1.1.1 port 8888 ssh2: RSA SHA256:foobar","procid":1111,"timestamp":"2025-03-23T01:49:58Z"}
Keys and values can be wrapped using the " character.
" characters can be escaped by the \ character.
As per this logfmt specification, the parse_logfmt function
accepts standalone keys and assigns them a Boolean value of true.
argument
Tipo
Descripción
default
required
value
string
The string to parse.
N/A
yes
Errors
value is not a properly formatted key-value string
Examples
Parse logfmt log
Source:
parse_logfmt!("@timestamp=\"Sun Jan 10 16:47:39 EST 2021\" level=info msg=\"Stopping all fetchers\" tag#production=stopping_fetchers id=ConsumerFetcherManager-1382721708341 module=kafka.consumer.ConsumerFetcherManager")
Return:
{"@timestamp":"Sun Jan 10 16:47:39 EST 2021","level":"info","msg":"Stopping all fetchers","tag#production":"stopping_fetchers","id":"ConsumerFetcherManager-1382721708341","module":"kafka.consumer.ConsumerFetcherManager"}
parse_nginx_log
Parses Nginx access and error log lines. Lines can be in [`combined`](https://nginx.org/en/docs/http/ngx_http_log_module.html),
[`ingress_upstreaminfo`](https://kubernetes.github.io/ingress-nginx/user-guide/nginx-configuration/log-format/), [`main`](https://hg.nginx.org/pkg-oss/file/tip/debian/debian/nginx.conf) or [`error`](https://github.com/nginx/nginx/blob/branches/stable-1.18/src/core/ngx_log.c#L102) format.
argument
Tipo
Descripción
default
required
value
string
The string to parse.
N/A
yes
timestamp_format
string
The date/time format to use for encoding the timestamp. The time is parsed
in local time if the timestamp doesn’t specify a timezone. The default format is %d/%b/%Y:%T %z for
combined logs and %Y/%m/%d %H:%M:%S for error logs.
%d/%b/%Y:%T %z
no
format
string
The format to use for parsing the log.
N/A
yes
Errors
value does not match the specified format.
timestamp_format is not a valid format string.
The timestamp in value fails to parse using the provided timestamp_format.
Examples
Parse via Nginx log format (combined)
Source:
parse_nginx_log!( s'172.17.0.1 - alice [01/Apr/2021:12:02:31 +0000] "POST /not-found HTTP/1.1" 404 153 "http://localhost/somewhere" "Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/72.0.3626.119 Safari/537.36" "2.75"',
"combined",
)
Return:
{"agent":"Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/72.0.3626.119 Safari/537.36","client":"172.17.0.1","compression":"2.75","referer":"http://localhost/somewhere","request":"POST /not-found HTTP/1.1","size":153,"status":404,"timestamp":"2021-04-01T12:02:31Z","user":"alice"}
Parse via Nginx log format (error)
Source:
parse_nginx_log!( s'2021/04/01 13:02:31 [error] 31#31: *1 open() "/usr/share/nginx/html/not-found" failed (2: No such file or directory), client: 172.17.0.1, server: localhost, request: "POST /not-found HTTP/1.1", host: "localhost:8081"',
"error")
Return:
{"timestamp":"2021-04-01T13:02:31Z","severity":"error","pid":31,"tid":31,"cid":1,"message":"open() \"/usr/share/nginx/html/not-found\" failed (2: No such file or directory)","client":"172.17.0.1","server":"localhost","request":"POST /not-found HTTP/1.1","host":"localhost:8081"}
Parse via Nginx log format (ingress_upstreaminfo)
Source:
parse_nginx_log!( s'0.0.0.0 - bob [18/Mar/2023:15:00:00 +0000] "GET /some/path HTTP/2.0" 200 12312 "https://10.0.0.1/some/referer" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/111.0.0.0 Safari/537.36" 462 0.050 [some-upstream-service-9000] [some-other-upstream-5000] 10.0.50.80:9000 19437 0.049 200 752178adb17130b291aefd8c386279e7',
"ingress_upstreaminfo")
Return:
{"body_bytes_size":12312,"http_referer":"https://10.0.0.1/some/referer","http_user_agent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/111.0.0.0 Safari/537.36","proxy_alternative_upstream_name":"some-other-upstream-5000","proxy_upstream_name":"some-upstream-service-9000","remote_addr":"0.0.0.0","remote_user":"bob","req_id":"752178adb17130b291aefd8c386279e7","request":"GET /some/path HTTP/2.0","request_length":462,"request_time":0.05,"status":200,"timestamp":"2023-03-18T15:00:00Z","upstream_addr":"10.0.50.80:9000","upstream_response_length":19437,"upstream_response_time":0.049,"upstream_status":200}
parse_syslog!( s'<13>1 2020-03-13T20:45:38.119Z dynamicwireless.name non 2426 ID931 [exampleSDID@32473 iut="3" eventSource= "Application" eventID="1011"] Try to override the THX port, maybe it will reboot the neural interface!')
Return:
{"severity":"notice","facility":"user","timestamp":"2020-03-13T20:45:38.119Z","hostname":"dynamicwireless.name","appname":"non","procid":2426,"msgid":"ID931","message":"Try to override the THX port, maybe it will reboot the neural interface!","exampleSDID@32473":{"eventID":"1011","eventSource":"Application","iut":"3"},"version":1}
The TZ database format. By default, this function parses the timestamp by global timezone option.
This argument overwrites the setting and is useful for parsing timestamps without a specified timezone, such as 16/10/2019 12:00:00.
If true and the port number is not specified in the input URL
string (or matches the default port for the scheme), it is
populated from well-known ports for the following schemes:
http, https, ws, wss, and ftp.
parse_user_agent("Opera/9.80 (J2ME/MIDP; Opera Mini/4.3.24214; iPhone; CPU iPhone OS 4_2_1 like Mac OS X; AppleWebKit/24.783; U; en) Presto/2.5.25 Version/10.54",
mode: "enriched")
{"book":{"@category":"CHILDREN","author":"J K. Rowling","title":{"@lang":"en","value":"Harry Potter"},"year":"2005"}}
parse_yaml
Parses the value as YAML.
argument
Tipo
Descripción
default
required
value
string
The string representation of the YAML to parse.
N/A
yes
Errors
value is not a valid YAML-formatted payload.
Examples
Parse YAML
Source:
parse_yaml!("key: val")
Return:
{"key":"val"}
Path Functions
del
Removes the field specified by the static path from the target.
For dynamic path deletion, see the remove function.
argument
Tipo
Descripción
default
required
path
path
The path of the field to delete.
N/A
yes
compact
boolean
After deletion, if compact is true and there is an empty object or array left,
the empty object or array is also removed, cascading up to the root. This only
applies to the path being deleted, and any parent paths.
false
no
Examples
Delete a field
Source:
del(.field1)
Rename a field
Source:
.new_field = del(.old_field)
exists
Checks whether the path exists for the target.
This function distinguishes between a missing path
and a path with a null value. A regular path lookup,
such as .foo, cannot distinguish between the two cases
since it always returns null if the path doesn’t exist.
argument
Tipo
Descripción
default
required
path
path
The path of the field to check.
N/A
yes
Examples
Exists (field)
Source:
exists(.field)
Return:
true
Exists (array element)
Source:
exists(.array[2])
Return:
true
get
Dynamically get the value of a given path.
If you know the path you want to look up, use
static paths such as .foo.bar[1] to get the value of that
path. However, if you do not know the path names,
use the dynamic get function to get the requested
value.
If you know the path you want to remove, use
the del function and static paths such as del(.foo.bar[1])
to remove the value at that path. The del function returns the
deleted value, and is more performant than remove.
However, if you do not know the path names, use the dynamic
remove function to remove the value at the provided path.
argument
Tipo
Descripción
default
required
value
object, array
The object or array to remove data from.
N/A
yes
path
array
An array of path segments to remove the value from.
N/A
yes
compact
boolean
After deletion, if compact is true, any empty objects or
arrays left are also removed.
Dynamically insert data into the path of a given object or array.
If you know the path you want to assign a value to,
use static path assignments such as .foo.bar[1] = true for
improved performance and readability. However, if you do not
know the path names, use the dynamic set function to
insert the data into the object or array.
argument
Tipo
Descripción
default
required
value
object, array
The object or array to insert data into.
N/A
yes
path
array
An array of path segments to insert the value into.
Takes the value string, and turns it into camelCase. Optionally, you can
pass in the existing case of the function, or else an attempt is made to determine the case automatically.
argument
Tipo
Descripción
default
required
value
string
The string to convert to camelCase.
N/A
yes
original_case
string
Optional hint on the original case type. Must be one of: kebab-case, camelCase, PascalCase, SCREAMING_SNAKE, snake_case
Takes the value string, and turns it into kebab-case. Optionally, you can
pass in the existing case of the function, or else we will try to figure out the case automatically.
argument
Tipo
Descripción
default
required
value
string
The string to convert to kebab-case.
N/A
yes
original_case
string
Optional hint on the original case type. Must be one of: kebab-case, camelCase, PascalCase, SCREAMING_SNAKE, snake_case
N/A
no
Examples
kebab-case a string
Source:
kebabcase("InputString")
Return:
"input-string"
kebab-case a string
Source:
kebabcase("InputString", "PascalCase")
Return:
"input-string"
match
Determines whether the value matches the pattern.
argument
Tipo
Descripción
default
required
value
string
The value to match.
N/A
yes
pattern
regex
The regular expression pattern to match against.
N/A
yes
Examples
Regex match on a string
Source:
match("I'm a little teapot", r'teapot')
Return:
true
String does not match the regular expression
Source:
match("I'm a little teapot", r'.*balloon')
Return:
false
match_any
Determines whether value matches any of the given patterns. All
patterns are checked in a single pass over the target string, giving this
function a potential performance advantage over the multiple calls
in the match function.
argument
Tipo
Descripción
default
required
value
string
The value to match.
N/A
yes
patterns
array
The array of regular expression patterns to match against.
N/A
yes
Examples
Regex match on a string
Source:
match_any("I'm a little teapot", [r'frying pan', r'teapot'])
Return:
true
parse_float
Parses the string value representing a floating point number in base 10 to a float.
argument
Tipo
Descripción
default
required
value
string
The string to parse.
N/A
yes
Errors
value is not a string.
Examples
Parse negative integer
Source:
parse_float!("-42")
Return:
-42
Parse negative integer
Source:
parse_float!("42.38")
Return:
42.38
Scientific notation
Source:
parse_float!("2.5e3")
Return:
2500
pascalcase
Takes the value string, and turns it into PascalCase. Optionally, you can
pass in the existing case of the function, or else we will try to figure out the case automatically.
argument
Tipo
Descripción
default
required
value
string
The string to convert to PascalCase.
N/A
yes
original_case
string
Optional hint on the original case type. Must be one of: kebab-case, camelCase, PascalCase, SCREAMING_SNAKE, snake_case
Other forms of personally identifiable information with custom patterns
This can help achieve compliance by ensuring sensitive data does not leave your network.
argument
Tipo
Descripción
default
required
value
string, object, array
The value to redact sensitive data from.
The function’s behavior depends on value’s type:
For strings, the sensitive data is redacted and a new string is returned.
For arrays, the sensitive data is redacted in each string element.
For objects, the sensitive data in each string value is masked, but the keys are not masked.
For arrays and objects, the function recurses into any nested arrays or objects. Any non-string elements are
skipped.
Redacted text is replaced with [REDACTED].
N/A
yes
filters
array
List of filters applied to value.
Each filter can be specified in the following ways:
As a regular expression, which is used to redact text that match it.
As an object with a type key that corresponds to a named filter and additional keys for customizing that filter.
As a named filter, if it has no required parameters.
Named filters can be a:
pattern: Redacts text matching any regular expressions specified in the patterns
key, which is required. This is the expanded version of just passing a regular expression as a filter.
us_social_security_number: Redacts US social security card numbers.
See examples for more details.
This parameter must be a static expression so that the argument can be validated at compile-time
to avoid runtime errors. You cannot use variables or other dynamic expressions with it.
N/A
yes
redactor
string, object
Specifies what to replace the redacted strings with.
It is given as an object with a “type” key specifying the type of redactor to use
and additional keys depending on the type. The following types are supported:
full: The default. Replace with the string “[REDACTED]”.
text: Replace with a custom string. The replacement key is required, and must
contain the string that is used as a replacement.
sha2: Hash the redacted text with SHA-2 as with sha2. Supports two optional parameters:
variant: The variant of the algorithm to use. Defaults to SHA-512/256.
encoding: How to encode the hash as text. Can be base16 or base64.
Defaults to base64.
sha3: Hash the redacted text with SHA-3 as with sha3. Supports two optional parameters:
variant: The variant of the algorithm to use. Defaults to SHA3-512.
encoding: How to encode the hash as text. Can be base16 or base64.
Defaults to base64.
As a convenience you can use a string as a shorthand for common redactor patterns:
"full" is equivalent to {"type": "full"}
"sha2" is equivalent to {"type": "sha2", "variant": "SHA-512/256", "encoding": "base64"}
"sha3" is equivalent to {"type": "sha3", "variant": "SHA3-512", "encoding": "base64"}
This parameter must be a static expression so that the argument can be validated at compile-time
to avoid runtime errors. You cannot use variables or other dynamic expressions with it.
redact("my id is 123456", filters: [r'\d+'], redactor: {"type": "text", "replacement": "***"})
Return:
"my id is ***"
Replace with SHA-2 hash
Source:
redact("my id is 123456", filters: [r'\d+'], redactor: "sha2")
Return:
"my id is GEtTedW1p6tC094dDKH+3B8P+xSnZz69AmpjaXRd63I="
Replace with SHA-3 hash
Source:
redact("my id is 123456", filters: [r'\d+'], redactor: "sha3")
Return:
"my id is ZNCdmTDI7PeeUTFnpYjLdUObdizo+bIupZdl8yqnTKGdLx6X3JIqPUlUWUoFBikX+yTR+OcvLtAqWO11NPlNJw=="
Replace with SHA-256 hash using hex encoding
Source:
redact("my id is 123456", filters: [r'\d+'], redactor: {"type": "sha2", "variant": "SHA-256", "encoding": "base16"})
Return:
"my id is 8d969eef6ecad3c29a3a629280e686cf0c3f5d5a86aff3ca12020c923adc6c92"
replace
Replaces all matching instances of pattern in value.
The pattern argument accepts regular expression capture groups.
Note when using capture groups:
You will need to escape the $ by using $$ to avoid Vector interpreting it as an
environment variable when loading configuration
If you want a literal $ in the replacement pattern, you will also need to escape this
with $$. When combined with environment variable interpolation in config files this
means you will need to use $$$$ to have a literal $ in the replacement pattern.
argument
Tipo
Descripción
default
required
value
string
The original string.
N/A
yes
pattern
regex, string
Replace all matches of this pattern. Can be a static string or a regular expression.
N/A
yes
with
string
The string that the matches are replaced with.
N/A
yes
count
integer
The maximum number of replacements to perform. -1 means replace all matches.
-1
no
Examples
Replace literal text
Source:
replace("Apples and Bananas", "and", "not")
Return:
"Apples not Bananas"
Replace using regular expression
Source:
replace("Apples and Bananas", r'(?i)bananas', "Pineapples")
Return:
"Apples and Pineapples"
Replace first instance
Source:
replace("Bananas and Bananas", "Bananas", "Pineapples", count: 1)
Return:
"Pineapples and Bananas"
Replace with capture groups (Note: Use $$num in config files)
Takes the value string, and turns it into SCREAMING_SNAKE case. Optionally, you can
pass in the existing case of the function, or else we will try to figure out the case automatically.
argument
Tipo
Descripción
default
required
value
string
The string to convert to SCREAMING_SNAKE case.
N/A
yes
original_case
string
Optional hint on the original case type. Must be one of: kebab-case, camelCase, PascalCase, SCREAMING_SNAKE, snake_case
N/A
no
Examples
SCREAMING_SNAKE a string
Source:
screamingsnakecase("input-string")
Return:
"INPUT_STRING"
SCREAMING_SNAKE a string
Source:
screamingsnakecase("input-string", "kebab-case")
Return:
"INPUT_STRING"
shannon_entropy
Generates Shannon entropy from given string. It can generate it
based on string bytes, codepoints, or graphemes.
argument
Tipo
Descripción
default
required
value
string
The input string.
N/A
yes
segmentation
string
Defines how to split the string to calculate entropy, based on occurrences of
segments.
Byte segmentation is the fastest, but it might give undesired results when handling
UTF-8 strings, while grapheme segmentation is the slowest, but most correct in these
cases.
Returns a slice of value between the start and end positions.
If the start and end parameters are negative, they refer to positions counting from the right of the
string or array. If end refers to a position that is greater than the length of the string or array,
a slice up to the end of the string or array is returned.
argument
Tipo
Descripción
default
required
value
array, string
The string or array to slice.
N/A
yes
start
integer
The inclusive start position. A zero-based index that can be negative.
N/A
yes
end
integer
The exclusive end position. A zero-based index that can be negative.
Takes the value string, and turns it into snake-case. Optionally, you can
pass in the existing case of the function, or else we will try to figure out the case automatically.
argument
Tipo
Descripción
default
required
value
string
The string to convert to snake-case.
N/A
yes
original_case
string
Optional hint on the original case type. Must be one of: kebab-case, camelCase, PascalCase, SCREAMING_SNAKE, snake_case
N/A
no
Examples
snake-case a string
Source:
snakecase("input-string")
Return:
"input_string"
snake-case a string
Source:
snakecase("input-string", "kebab-case")
Return:
"input_string"
split
Splits the value string using pattern.
argument
Tipo
Descripción
default
required
value
string
The string to split.
N/A
yes
pattern
string, regex
The string is split whenever this pattern is matched.
N/A
yes
limit
integer
The maximum number of substrings to return.
N/A
no
Examples
Split a string (no limit)
Source:
split("apples and pears and bananas", " and ")
Return:
["apples","pears","bananas"]
Split a string (with a limit)
Source:
split("apples and pears and bananas", " and ", limit: 2)
Return:
["apples","pears and bananas"]
starts_with
Determines whether value begins with substring.
argument
Tipo
Descripción
default
required
value
string
The string to search.
N/A
yes
substring
string
The substring that the value must start with.
N/A
yes
case_sensitive
boolean
Whether the match should be case sensitive.
true
no
Examples
String starts with (case sensitive)
Source:
starts_with("The Needle In The Haystack", "The Needle")
Return:
true
String starts with (case insensitive)
Source:
starts_with("The Needle In The Haystack", "the needle", case_sensitive: false)
Strips whitespace from the start and end of value, where whitespace is defined by the Unicode
White_Space property.
argument
Tipo
Descripción
default
required
value
string
The string to trim.
N/A
yes
Examples
Strip whitespace
Source:
strip_whitespace(" A sentence. ")
Return:
"A sentence."
truncate
Truncates the value string up to the limit number of characters.
argument
Tipo
Descripción
default
required
value
string
The string to truncate.
N/A
yes
limit
integer, float
The number of characters to truncate the string after.
N/A
yes
ellipsis
boolean
This argument is deprecated. An ellipsis (...) is appended if the parameter is set to trueand the value string
is truncated because it exceeded the limit.
N/A
no
suffix
string
A custom suffix (...) is appended to truncated strings.
If ellipsis is set to true, this parameter is ignored for backwards compatibility.
N/A
no
Examples
Truncate a string
Source:
truncate("A rather long sentence.", limit: 11, suffix: "...")
Return:
"A rather lo..."
Truncate a string
Source:
truncate("A rather long sentence.", limit: 11, suffix: "[TRUNCATED]")
Return:
"A rather lo[TRUNCATED]"
upcase
Upcases value, where upcase is defined according to the Unicode Derived Core Property
Uppercase.
argument
Tipo
Descripción
default
required
value
string
The string to convert to uppercase.
N/A
yes
Examples
Upcase a string
Source:
upcase("Hello, World!")
Return:
"HELLO, WORLD!"
System Functions
get_hostname
Returns the local system’s hostname.
Errors
Internal hostname resolution failed.
Examples
Get hostname
Source:
.hostname = get_hostname!()
get_timezone_name
Returns the name of the timezone in the Vector configuration (see
global configuration options).
If the configuration is set to local, then it attempts to
determine the name of the timezone from the host OS. If this
is not possible, then it returns the fixed offset of the
local timezone for the current time in the format "[+-]HH:MM",
for example, "+02:00".
Errors
Retrieval of local timezone information failed.
Examples
Get the IANA name of Vector’s timezone
Source:
.vector_timezone = get_timezone_name!()
Timestamp Functions
format_timestamp
Formats value into a string representation of the timestamp.
Returns the current timestamp in the UTC timezone with nanosecond precision.
Examples
Generate a current timestamp
Source:
now()
Return:
"2021-03-04T10:51:15.928937Z"
Type Functions
array
Returns value if it is an array, otherwise returns an error. This enables the type checker to guarantee that the
returned value is an array and can be used in any function that expects an array.
argument
Tipo
Descripción
default
required
value
any
The value to check if it is an array.
N/A
yes
Errors
value is not an array.
Examples
Declare an array type
Source:
array!(.value)
Return:
[1,2,3]
bool
Returns value if it is a Boolean, otherwise returns an error. This enables the type checker to guarantee that the
returned value is a Boolean and can be used in any function that expects a Boolean.
argument
Tipo
Descripción
default
required
value
any
The value to check if it is a Boolean.
N/A
yes
Errors
value is not a Boolean.
Examples
Declare a Boolean type
Source:
bool!(.value)
Return:
false
float
Returns value if it is a float, otherwise returns an error. This enables the type checker to guarantee that the
returned value is a float and can be used in any function that expects a float.
argument
Tipo
Descripción
default
required
value
any
The value to check if it is a float.
N/A
yes
Errors
value is not a float.
Examples
Declare a float type
Source:
float!(.value)
Return:
42
int
Returns value if it is an integer, otherwise returns an error. This enables the type checker to guarantee that the
returned value is an integer and can be used in any function that expects an integer.
argument
Tipo
Descripción
default
required
value
any
The value to check if it is an integer.
N/A
yes
Errors
value is not an integer.
Examples
Declare an integer type
Source:
int!(.value)
Return:
42
is_array
Check if the value’s type is an array.
argument
Tipo
Descripción
default
required
value
any
The value to check if it is an array.
N/A
yes
Examples
Valid array
Source:
is_array([1, 2, 3])
Return:
true
Non-matching type
Source:
is_array("a string")
Return:
false
is_boolean
Check if the value’s type is a boolean.
argument
Tipo
Descripción
default
required
value
any
The value to check if it is a Boolean.
N/A
yes
Examples
Valid boolean
Source:
is_boolean(false)
Return:
true
Non-matching type
Source:
is_boolean("a string")
Return:
false
is_empty
Check if the object, array, or string has a length of 0.
argument
Tipo
Descripción
default
required
value
object, array, string
The value to check.
N/A
yes
Examples
Empty array
Source:
is_empty([])
Return:
true
Non-empty string
Source:
is_empty("a string")
Return:
false
Non-empty object
Source:
is_empty({"foo": "bar"})
Return:
false
is_float
Check if the value’s type is a float.
argument
Tipo
Descripción
default
required
value
any
The value to check if it is a float.
N/A
yes
Examples
Valid float
Source:
is_float(0.577)
Return:
true
Non-matching type
Source:
is_float("a string")
Return:
false
is_integer
Check if the value`’s type is an integer.
argument
Tipo
Descripción
default
required
value
any
The value to check if it is an integer.
N/A
yes
Examples
Valid integer
Source:
is_integer(1)
Return:
true
Non-matching type
Source:
is_integer("a string")
Return:
false
is_json
Check if the string is a valid JSON document.
argument
Tipo
Descripción
default
required
value
string
The value to check if it is a valid JSON document.
N/A
yes
variant
string
The variant of the JSON type to explicitly check for.
N/A
no
Examples
Valid JSON object
Source:
is_json("{}")
Return:
true
Non-valid value
Source:
is_json("{")
Return:
false
Exact variant
Source:
is_json("{}", variant: "object")
Return:
true
Non-valid exact variant
Source:
is_json("{}", variant: "array")
Return:
false
is_null
Check if value’s type is null. For a more relaxed function,
see is_nullish.
argument
Tipo
Descripción
default
required
value
any
The value to check if it is null.
N/A
yes
Examples
Null value
Source:
is_null(null)
Return:
true
Non-matching type
Source:
is_null("a string")
Return:
false
is_nullish
Determines whether value is nullish. Returns true if the specified value is null,
an empty string, a string containing only whitespace, or the string "-". Returns false otherwise.
argument
Tipo
Descripción
default
required
value
any
The value to check for nullishness, for example, a useless value.
N/A
yes
Examples
Null detection (blank string)
Source:
is_nullish("")
Return:
true
Null detection (dash string)
Source:
is_nullish("-")
Return:
true
Null detection (whitespace)
Source:
is_nullish("
")
Return:
true
is_object
Check if value’s type is an object.
argument
Tipo
Descripción
default
required
value
any
The value to check if it is an object.
N/A
yes
Examples
Valid object
Source:
is_object({"foo": "bar"})
Return:
true
Non-matching type
Source:
is_object("a string")
Return:
false
is_regex
Check if value’s type is a regex.
argument
Tipo
Descripción
default
required
value
any
The value to check if it is a regex.
N/A
yes
Examples
Valid regex
Source:
is_regex(r'pattern')
Return:
true
Non-matching type
Source:
is_regex("a string")
Return:
false
is_string
Check if value’s type is a string.
argument
Tipo
Descripción
default
required
value
any
The value to check if it is a string.
N/A
yes
Examples
Valid string
Source:
is_string("a string")
Return:
true
Non-matching type
Source:
is_string([1, 2, 3])
Return:
false
is_timestamp
Check if value’s type is a timestamp.
argument
Tipo
Descripción
default
required
value
any
The value to check if it is a timestamp.
N/A
yes
Examples
Valid timestamp
Source:
is_timestamp(t'2021-03-26T16:00:00Z')
Return:
true
Non-matching type
Source:
is_timestamp("a string")
Return:
false
object
Returns value if it is an object, otherwise returns an error. This enables the type checker to guarantee that the
returned value is an object and can be used in any function that expects an object.
argument
Tipo
Descripción
default
required
value
any
The value to check if it is an object.
N/A
yes
Errors
value is not an object.
Examples
Declare an object type
Source:
object!(.value)
Return:
{"field1":"value1","field2":"value2"}
string
Returns value if it is a string, otherwise returns an error. This enables the type checker to guarantee that the
returned value is a string and can be used in any function that expects a string.
argument
Tipo
Descripción
default
required
value
any
The value to check if it is a string.
N/A
yes
Errors
value is not a string.
Examples
Declare a string type
Source:
string!(.message)
Return:
"{\"field\": \"value\"}"
tag_types_externally
Adds type information to all (nested) scalar values in the provided value.
The type information is added externally, meaning that value has the form of "type": value after this
transformation.
Returns value if it is a timestamp, otherwise returns an error. This enables the type checker to guarantee that
the returned value is a timestamp and can be used in any function that expects a timestamp.
