This product is not supported for your selected Datadog site. ().

gcp_eventarc_pipeline

ancestors

Type: UNORDERED_LIST_STRING

annotations

Type: MAP_STRING_STRING
Provider name: annotations
Description: Optional. User-defined annotations. See https://google.aip.dev/128#annotations.

create_time

Type: TIMESTAMP
Provider name: createTime
Description: Output only. The creation time. A timestamp in RFC3339 UTC “Zulu” format, with nanosecond resolution and up to nine fractional digits. Examples: “2014-10-02T15:01:23Z” and “2014-10-02T15:01:23.045123456Z”.

crypto_key_name

Type: STRING
Provider name: cryptoKeyName
Description: Optional. Resource name of a KMS crypto key (managed by the user) used to encrypt/decrypt the event data. If not set, an internal Google-owned key will be used to encrypt messages. It must match the pattern “projects/{project}/locations/{location}/keyRings/{keyring}/cryptoKeys/{key}”.

destinations

Type: UNORDERED_LIST_STRUCT
Provider name: destinations
Description: Required. List of destinations to which messages will be forwarded. Currently, exactly one destination is supported per Pipeline.

  • authentication_config
    Type: STRUCT
    Provider name: authenticationConfig
    Description: Optional. An authentication config used to authenticate message requests, such that destinations can verify the source. For example, this can be used with private GCP destinations that require GCP credentials to access like Cloud Run. This field is optional and should be set only by users interested in authenticated push
    • google_oidc
      Type: STRUCT
      Provider name: googleOidc
      Description: Optional. This authenticate method will apply Google OIDC tokens signed by a GCP service account to the requests.
      • audience
        Type: STRING
        Provider name: audience
        Description: Optional. Audience to be used to generate the OIDC Token. The audience claim identifies the recipient that the JWT is intended for. If unspecified, the destination URI will be used.
      • service_account
        Type: STRING
        Provider name: serviceAccount
        Description: Required. Service account email used to generate the OIDC Token. The principal who calls this API must have iam.serviceAccounts.actAs permission in the service account. See https://cloud.google.com/iam/docs/understanding-service-accounts for more information. Eventarc service agents must have roles/roles/iam.serviceAccountTokenCreator role to allow the Pipeline to create OpenID tokens for authenticated requests.
    • oauth_token
      Type: STRUCT
      Provider name: oauthToken
      Description: Optional. If specified, an OAuth token will be generated and attached as an Authorization header in the HTTP request. This type of authorization should generally only be used when calling Google APIs hosted on *.googleapis.com.
      • scope
        Type: STRING
        Provider name: scope
        Description: Optional. OAuth scope to be used for generating OAuth access token. If not specified, “https://www.googleapis.com/auth/cloud-platform" will be used.
      • service_account
        Type: STRING
        Provider name: serviceAccount
        Description: Required. Service account email used to generate the OAuth token. The principal who calls this API must have iam.serviceAccounts.actAs permission in the service account. See https://cloud.google.com/iam/docs/understanding-service-accounts for more information. Eventarc service agents must have roles/roles/iam.serviceAccountTokenCreator role to allow Pipeline to create OAuth2 tokens for authenticated requests.
  • http_endpoint
    Type: STRUCT
    Provider name: httpEndpoint
    Description: Optional. An HTTP endpoint destination described by an URI. If a DNS FQDN is provided as the endpoint, Pipeline will create a peering zone to the consumer VPC and forward DNS requests to the VPC specified by network config to resolve the service endpoint. See: https://cloud.google.com/dns/docs/zones/zones-overview#peering_zones
    • message_binding_template
      Type: STRING
      Provider name: messageBindingTemplate
      Description: Optional. The CEL expression used to modify how the destination-bound HTTP request is constructed. If a binding expression is not specified here, the message is treated as a CloudEvent and is mapped to the HTTP request according to the CloudEvent HTTP Protocol Binding Binary Content Mode (https://github.com/cloudevents/spec/blob/main/cloudevents/bindings/http-protocol-binding.md#31-binary-content-mode). In this representation, all fields except the data and datacontenttype field on the message are mapped to HTTP request headers with a prefix of ce-. To construct the HTTP request payload and the value of the content-type HTTP header, the payload format is defined as follows: 1) Use the output_payload_format_type on the Pipeline.Destination if it is set, else: 2) Use the input_payload_format_type on the Pipeline if it is set, else: 3) Treat the payload as opaque binary data. The data field of the message is converted to the payload format or left as-is for case 3) and then attached as the payload of the HTTP request. The content-type header on the HTTP request is set to the payload format type or left empty for case 3). However, if a mediation has updated the datacontenttype field on the message so that it is not the same as the payload format type but it is still a prefix of the payload format type, then the content-type header on the HTTP request is set to this datacontenttype value. For example, if the datacontenttype is “application/json” and the payload format type is “application/json; charset=utf-8”, then the content-type header on the HTTP request is set to “application/json; charset=utf-8”. If a non-empty binding expression is specified then this expression is used to modify the default CloudEvent HTTP Protocol Binding Binary Content representation. The result of the CEL expression must be a map of key/value pairs which is used as follows: - If a map named headers exists on the result of the expression, then its key/value pairs are directly mapped to the HTTP request headers. The headers values are constructed from the corresponding value type’s canonical representation. If the headers field doesn’t exist then the resulting HTTP request will be the headers of the CloudEvent HTTP Binding Binary Content Mode representation of the final message. Note: If the specified binding expression, has updated the datacontenttype field on the message so that it is not the same as the payload format type but it is still a prefix of the payload format type, then the content-type header in the headers map is set to this datacontenttype value. - If a field named body exists on the result of the expression then its value is directly mapped to the body of the request. If the value of the body field is of type bytes or string then it is used for the HTTP request body as-is, with no conversion. If the body field is of any other type then it is converted to a JSON string. If the body field does not exist then the resulting payload of the HTTP request will be data value of the CloudEvent HTTP Binding Binary Content Mode representation of the final message as described earlier. - Any other fields in the resulting expression will be ignored. The CEL expression may access the incoming CloudEvent message in its definition, as follows: - The data field of the incoming CloudEvent message can be accessed using the message.data value. Subfields of message.data may also be accessed if an input_payload_format has been specified on the Pipeline. - Each attribute of the incoming CloudEvent message can be accessed using the message. value, where is replaced with the name of the attribute. - Existing headers can be accessed in the CEL expression using the headers variable. The headers variable defines a map of key/value pairs corresponding to the HTTP headers of the CloudEvent HTTP Binding Binary Content Mode representation of the final message as described earlier. For example, the following CEL expression can be used to construct an HTTP request by adding an additional header to the HTTP headers of the CloudEvent HTTP Binding Binary Content Mode representation of the final message and by overwriting the body of the request: { "headers": headers.merge({"new-header-key": "new-header-value"}), "body": "new-body" } - The default binding for the message payload can be accessed using the body variable. It conatins a string representation of the message payload in the format specified by the output_payload_format field. If the input_payload_format field is not set, the body variable contains the same message payload bytes that were published. Additionally, the following CEL extension functions are provided for use in this CEL expression: - toBase64Url: map.toBase64Url() -> string - Converts a CelValue to a base64url encoded string - toJsonString: map.toJsonString() -> string - Converts a CelValue to a JSON string - merge: map1.merge(map2) -> map3 - Merges the passed CEL map with the existing CEL map the function is applied to. - If the same key exists in both maps, if the key’s value is type map both maps are merged else the value from the passed map is used. - denormalize: map.denormalize() -> map - Denormalizes a CEL map such that every value of type map or key in the map is expanded to return a single level map. - The resulting keys are “.” separated indices of the map keys. - For example: { “a”: 1, “b”: { “c”: 2, “d”: 3 } “e”: [4, 5] } .denormalize() -> { “a”: 1, “b.c”: 2, “b.d”: 3, “e.0”: 4, “e.1”: 5 } - setField: map.setField(key, value) -> message - Sets the field of the message with the given key to the given value. - If the field is not present it will be added. - If the field is present it will be overwritten. - The key can be a dot separated path to set a field in a nested message. - Key must be of type string. - Value may be any valid type. - removeFields: map.removeFields([key1, key2, …]) -> message - Removes the fields of the map with the given keys. - The keys can be a dot separated path to remove a field in a nested message. - If a key is not found it will be ignored. - Keys must be of type string. - toMap: [map1, map2, …].toMap() -> map - Converts a CEL list of CEL maps to a single CEL map - toCloudEventJsonWithPayloadFormat: message.toCloudEventJsonWithPayloadFormat() -> map - Converts a message to the corresponding structure of JSON format for CloudEvents. - It converts data to destination payload format specified in output_payload_format. If output_payload_format is not set, the data will remain unchanged. - It also sets the corresponding datacontenttype of the CloudEvent, as indicated by output_payload_format. If no output_payload_format is set it will use the value of the “datacontenttype” attribute on the CloudEvent if present, else remove “datacontenttype” attribute. - This function expects that the content of the message will adhere to the standard CloudEvent format. If it doesn’t then this function will fail. - The result is a CEL map that corresponds to the JSON representation of the CloudEvent. To convert that data to a JSON string it can be chained with the toJsonString function. The Pipeline expects that the message it receives adheres to the standard CloudEvent format. If it doesn’t then the outgoing message request may fail with a persistent error.
    • uri
      Type: STRING
      Provider name: uri
      Description: Required. The URI of the HTTP endpoint. The value must be a RFC2396 URI string. Examples: https://svc.us-central1.p.local:8080/route. Only the HTTPS protocol is supported.
  • message_bus
    Type: STRING
    Provider name: messageBus
    Description: Optional. The resource name of the Message Bus to which events should be published. The Message Bus resource should exist in the same project as the Pipeline. Format: projects/{project}/locations/{location}/messageBuses/{message_bus}
  • network_config
    Type: STRUCT
    Provider name: networkConfig
    Description: Optional. Network config is used to configure how Pipeline resolves and connects to a destination.
    • network_attachment
      Type: STRING
      Provider name: networkAttachment
      Description: Required. Name of the NetworkAttachment that allows access to the consumer VPC. Format: projects/{PROJECT_ID}/regions/{REGION}/networkAttachments/{NETWORK_ATTACHMENT_NAME}
  • output_payload_format
    Type: STRUCT
    Provider name: outputPayloadFormat
    Description: Optional. The message format before it is delivered to the destination. If not set, the message will be delivered in the format it was originally delivered to the Pipeline. This field can only be set if Pipeline.input_payload_format is also set.

    • avro
      Type: STRUCT
      Provider name: avro
      Description: Optional. Avro format.

      • schema_definition
        Type: STRING
        Provider name: schemaDefinition
        Description: Optional. The entire schema definition is stored in this field.

    • json
      Type: STRUCT
      Provider name: json
      Description: Optional. JSON format.

    • protobuf
      Type: STRUCT
      Provider name: protobuf
      Description: Optional. Protobuf format.

      • schema_definition
        Type: STRING
        Provider name: schemaDefinition
        Description: Optional. The entire schema definition is stored in this field.
  • topic
    Type: STRING
    Provider name: topic
    Description: Optional. The resource name of the Pub/Sub topic to which events should be published. Format: projects/{project}/locations/{location}/topics/{topic}
  • workflow
    Type: STRING
    Provider name: workflow
    Description: Optional. The resource name of the Workflow whose Executions are triggered by the events. The Workflow resource should be deployed in the same project as the Pipeline. Format: projects/{project}/locations/{location}/workflows/{workflow}

etag

Type: STRING
Provider name: etag
Description: Output only. This checksum is computed by the server based on the value of other fields, and might be sent only on create requests to ensure that the client has an up-to-date value before proceeding.

gcp_display_name

Type: STRING
Provider name: displayName
Description: Optional. Display name of resource.

input_payload_format

Type: STRUCT
Provider name: inputPayloadFormat
Description: Optional. The payload format expected for the messages received by the Pipeline. If input_payload_format is set then any messages not matching this format will be treated as persistent errors. If input_payload_format is not set, then the message data will be treated as an opaque binary and no output format can be set on the Pipeline through the Pipeline.Destination.output_payload_format field. Any Mediations on the Pipeline that involve access to the data field will fail as persistent errors.

  • avro
    Type: STRUCT
    Provider name: avro
    Description: Optional. Avro format.

    • schema_definition
      Type: STRING
      Provider name: schemaDefinition
      Description: Optional. The entire schema definition is stored in this field.

  • json
    Type: STRUCT
    Provider name: json
    Description: Optional. JSON format.

  • protobuf
    Type: STRUCT
    Provider name: protobuf
    Description: Optional. Protobuf format.

    • schema_definition
      Type: STRING
      Provider name: schemaDefinition
      Description: Optional. The entire schema definition is stored in this field.

labels

Type: UNORDERED_LIST_STRING

logging_config

Type: STRUCT
Provider name: loggingConfig
Description: Optional. Config to control Platform Logging for Pipelines.

  • log_severity
    Type: STRING
    Provider name: logSeverity
    Description: Optional. The minimum severity of logs that will be sent to Stackdriver/Platform Telemetry. Logs at severitiy ≥ this value will be sent, unless it is NONE.
    Possible values:
    • LOG_SEVERITY_UNSPECIFIED - Log severity is not specified. This value is treated the same as NONE, but is used to distinguish between no update and update to NONE in update_masks.
    • NONE - Default value at resource creation, presence of this value must be treated as no logging/disable logging.
    • DEBUG - Debug or trace level logging.
    • INFO - Routine information, such as ongoing status or performance.
    • NOTICE - Normal but significant events, such as start up, shut down, or a configuration change.
    • WARNING - Warning events might cause problems.
    • ERROR - Error events are likely to cause problems.
    • CRITICAL - Critical events cause more severe problems or outages.
    • ALERT - A person must take action immediately.
    • EMERGENCY - One or more systems are unusable.

mediations

Type: UNORDERED_LIST_STRUCT
Provider name: mediations
Description: Optional. List of mediation operations to be performed on the message. Currently, only one Transformation operation is allowed in each Pipeline.

  • transformation
    Type: STRUCT
    Provider name: transformation
    Description: Optional. How the Pipeline is to transform messages
    • transformation_template
      Type: STRING
      Provider name: transformationTemplate
      Description: Optional. The CEL expression template to apply to transform messages. The following CEL extension functions are provided for use in this CEL expression: - merge: map1.merge(map2) -> map3 - Merges the passed CEL map with the existing CEL map the function is applied to. - If the same key exists in both maps, if the key’s value is type map both maps are merged else the value from the passed map is used. - denormalize: map.denormalize() -> map - Denormalizes a CEL map such that every value of type map or key in the map is expanded to return a single level map. - The resulting keys are “.” separated indices of the map keys. - For example: { “a”: 1, “b”: { “c”: 2, “d”: 3 } “e”: [4, 5] } .denormalize() -> { “a”: 1, “b.c”: 2, “b.d”: 3, “e.0”: 4, “e.1”: 5 } - setField: map.setField(key, value) -> message - Sets the field of the message with the given key to the given value. - If the field is not present it will be added. - If the field is present it will be overwritten. - The key can be a dot separated path to set a field in a nested message. - Key must be of type string. - Value may be any valid type. - removeFields: map.removeFields([key1, key2, …]) -> message - Removes the fields of the map with the given keys. - The keys can be a dot separated path to remove a field in a nested message. - If a key is not found it will be ignored. - Keys must be of type string. - toMap: [map1, map2, …].toMap() -> map - Converts a CEL list of CEL maps to a single CEL map - toDestinationPayloadFormat(): message.data.toDestinationPayloadFormat() -> string or bytes - Converts the message data to the destination payload format specified in Pipeline.Destination.output_payload_format - This function is meant to be applied to the message.data field. - If the destination payload format is not set, the function will return the message data unchanged. - toCloudEventJsonWithPayloadFormat: message.toCloudEventJsonWithPayloadFormat() -> map - Converts a message to the corresponding structure of JSON format for CloudEvents - This function applies toDestinationPayloadFormat() to the message data. It also sets the corresponding datacontenttype of the CloudEvent, as indicated by Pipeline.Destination.output_payload_format. If no output_payload_format is set it will use the existing datacontenttype on the CloudEvent if present, else leave datacontenttype absent. - This function expects that the content of the message will adhere to the standard CloudEvent format. If it doesn’t then this function will fail. - The result is a CEL map that corresponds to the JSON representation of the CloudEvent. To convert that data to a JSON string it can be chained with the toJsonString function.

name

Type: STRING
Provider name: name
Description: Identifier. The resource name of the Pipeline. Must be unique within the location of the project and must be in projects/{project}/locations/{location}/pipelines/{pipeline} format.

organization_id

Type: STRING

parent

Type: STRING

project_id

Type: STRING

project_number

Type: STRING

resource_name

Type: STRING

retry_policy

Type: STRUCT
Provider name: retryPolicy
Description: Optional. The retry policy to use in the pipeline.

  • max_attempts
    Type: INT32
    Provider name: maxAttempts
    Description: Optional. The maximum number of delivery attempts for any message. The value must be between 1 and 100. The default value for this field is 5.
  • max_retry_delay
    Type: STRING
    Provider name: maxRetryDelay
    Description: Optional. The maximum amount of seconds to wait between retry attempts. The value must be between 1 and 600. The default value for this field is 60.
  • min_retry_delay
    Type: STRING
    Provider name: minRetryDelay
    Description: Optional. The minimum amount of seconds to wait between retry attempts. The value must be between 1 and 600. The default value for this field is 5.

satisfies_pzs

Type: BOOLEAN
Provider name: satisfiesPzs
Description: Output only. Whether or not this Pipeline satisfies the requirements of physical zone separation

tags

Type: UNORDERED_LIST_STRING

uid

Type: STRING
Provider name: uid
Description: Output only. Server-assigned unique identifier for the Pipeline. The value is a UUID4 string and guaranteed to remain unchanged until the resource is deleted.

update_time

Type: TIMESTAMP
Provider name: updateTime
Description: Output only. The last-modified time. A timestamp in RFC3339 UTC “Zulu” format, with nanosecond resolution and up to nine fractional digits. Examples: “2014-10-02T15:01:23Z” and “2014-10-02T15:01:23.045123456Z”.