Vertex AI PipelineJob

Docs > DDSQL Reference > Data Directory > Vertex AI PipelineJob

Vertex AI PipelineJob in Google Cloud is a managed resource that defines and executes machine learning workflows. It allows you to orchestrate complex ML pipelines composed of multiple steps, such as data preprocessing, training, evaluation, and deployment. PipelineJobs ensure reproducibility, scalability, and automation of ML processes, making it easier to manage end-to-end workflows.

gcp.aiplatform_pipeline_job

Fields

ID	Type	Data Type	Description
_key	core	string
ancestors	core	array<string>
create_time	core	timestamp	Output only. Pipeline creation time.
datadog_display_name	core	string
encryption_spec	core	json	Customer-managed encryption key spec for a pipelineJob. If set, this PipelineJob and all of its sub-resources will be secured by this key.
end_time	core	timestamp	Output only. Pipeline end time.
error	core	json	Output only. The error that occurred during pipeline execution. Only populated when the pipeline's state is FAILED or CANCELLED.
gcp_display_name	core	string	The display name of the Pipeline. The name can be up to 128 characters long and can consist of any UTF-8 characters.
job_detail	core	json	Output only. The details of pipeline run. Not available in the list view.
labels	core	array<string>	The labels with user-defined metadata to organize PipelineJob. Label keys and values can be no longer than 64 characters (Unicode codepoints), can only contain lowercase letters, numeric characters, underscores and dashes. International characters are allowed. See https://goo.gl/xmQnxf for more information and examples of labels. Note there is some reserved label key for Vertex AI Pipelines. - `vertex-ai-pipelines-run-billing-id`, user set value will get overrided.
name	core	string	Output only. The resource name of the PipelineJob.
network	core	string	The full name of the Compute Engine [network](/compute/docs/networks-and-firewalls#networks) to which the Pipeline Job's workload should be peered. For example, `projects/12345/global/networks/myVPC`. [Format](/compute/docs/reference/rest/v1/networks/insert) is of the form `projects/{project}/global/networks/{network}`. Where {project} is a project number, as in `12345`, and {network} is a network name. Private services access must already be configured for the network. Pipeline job will apply the network configuration to the Google Cloud resources being launched, if applied, such as Vertex AI Training or Dataflow job. If left unspecified, the workload is not peered with any network.
organization_id	core	string
parent	core	string
preflight_validations	core	bool	Optional. Whether to do component level validations before job creation.
project_id	core	string
project_number	core	string
reserved_ip_ranges	core	array<string>	A list of names for the reserved ip ranges under the VPC network that can be used for this Pipeline Job's workload. If set, we will deploy the Pipeline Job's workload within the provided ip ranges. Otherwise, the job will be deployed to any ip ranges under the provided VPC network. Example: ['vertex-ai-ip-range'].
resource_name	core	string
runtime_config	core	json	Runtime config of the pipeline.
schedule_name	core	string	Output only. The schedule resource name. Only returned if the Pipeline is created by Schedule API.
service_account	core	string	The service account that the pipeline workload runs as. If not specified, the Compute Engine default service account in the project will be used. See https://cloud.google.com/compute/docs/access/service-accounts#default_service_account Users starting the pipeline must have the `iam.serviceAccounts.actAs` permission on this service account.
start_time	core	timestamp	Output only. Pipeline start time.
state	core	string	Output only. The detailed state of the job.
tags	core	hstore
template_metadata	core	json	Output only. Pipeline template metadata. Will fill up fields if PipelineJob.template_uri is from supported template registry.
template_uri	core	string	A template uri from where the PipelineJob.pipeline_spec, if empty, will be downloaded. Currently, only uri from Vertex Template Registry & Gallery is supported. Reference to https://cloud.google.com/vertex-ai/docs/pipelines/create-pipeline-template.
update_time	core	timestamp	Output only. Timestamp when this PipelineJob was most recently updated.