Upload records to an Agent Observability dataset

v2 (latest)

Note: This endpoint is in preview and is subject to change. If you have any feedback, contact Datadog support.

POST https://api.ap1.datadoghq.com/api/v2/llm-obs/v2/{project_id}/datasets/{dataset_id}/records/uploadhttps://api.ap2.datadoghq.com/api/v2/llm-obs/v2/{project_id}/datasets/{dataset_id}/records/uploadhttps://api.datadoghq.eu/api/v2/llm-obs/v2/{project_id}/datasets/{dataset_id}/records/uploadhttps://api.ddog-gov.com/api/v2/llm-obs/v2/{project_id}/datasets/{dataset_id}/records/uploadhttps://api.us2.ddog-gov.com/api/v2/llm-obs/v2/{project_id}/datasets/{dataset_id}/records/uploadhttps://api.uk1.datadoghq.com/api/v2/llm-obs/v2/{project_id}/datasets/{dataset_id}/records/uploadhttps://api.datadoghq.com/api/v2/llm-obs/v2/{project_id}/datasets/{dataset_id}/records/uploadhttps://api.us3.datadoghq.com/api/v2/llm-obs/v2/{project_id}/datasets/{dataset_id}/records/uploadhttps://api.us5.datadoghq.com/api/v2/llm-obs/v2/{project_id}/datasets/{dataset_id}/records/upload

Overview

Upload records to a dataset from a file. The request is a multipart/form-data upload containing a single file part. Currently only CSV is supported. The CSV must include an input column. Optional columns are id, expected_output, metadata, and tags.

The response is a Server-Sent Events stream (text/event-stream) emitting progress updates while records are processed. The stream emits the following named events:

progress: incremental record counts written so far.
completed: terminal event with a JSON body containing records_created.
error: terminal event with a JSON body containing an error message.

Arguments

Path Parameters

Name

Type

Description

project_id [required]

string

The ID of the Agent Observability project.

dataset_id [required]

string

The ID of the Agent Observability dataset.

Query Strings

Name

Type

Description

deduplicate

boolean

Whether to skip records whose input already exists in the dataset. Defaults to false.

overwrite

boolean

Whether to overwrite existing records that share the same user-provided id. Defaults to true.

Request

Body Data (required)

Multipart upload payload containing the records file.

Expand All

Field

Type

Description

file

binary

The records file to upload. Currently only CSV is supported. The file must include an input column. Optional columns include id, expected_output, metadata, and tags.

{
  "file": "string"
}

Response

Bad Request

API error response.

Expand All

Field

Type

Description

errors [required]

[object]

A list of errors.

{
  "errors": [
    {
      "detail": "Missing required attribute in body",
      "meta": {},
      "source": {
        "header": "Authorization",
        "parameter": "limit",
        "pointer": "/data/attributes/title"
      },
      "status": "400",
      "title": "Bad Request"
    }
  ]
}

Unauthorized

API error response.

Expand All

Field

Type

Description

errors [required]

[object]

A list of errors.

{
  "errors": [
    {
      "detail": "Missing required attribute in body",
      "meta": {},
      "source": {
        "header": "Authorization",
        "parameter": "limit",
        "pointer": "/data/attributes/title"
      },
      "status": "400",
      "title": "Bad Request"
    }
  ]
}

Forbidden

API error response.

Expand All

Field

Type

Description

errors [required]

[object]

A list of errors.

{
  "errors": [
    {
      "detail": "Missing required attribute in body",
      "meta": {},
      "source": {
        "header": "Authorization",
        "parameter": "limit",
        "pointer": "/data/attributes/title"
      },
      "status": "400",
      "title": "Bad Request"
    }
  ]
}

Not Found

API error response.

Expand All

Field

Type

Description

errors [required]

[object]

A list of errors.

{
  "errors": [
    {
      "detail": "Missing required attribute in body",
      "meta": {},
      "source": {
        "header": "Authorization",
        "parameter": "limit",
        "pointer": "/data/attributes/title"
      },
      "status": "400",
      "title": "Bad Request"
    }
  ]
}

Too many requests

API error response.

Expand All

Field

Type

Description

errors [required]

[string]

A list of errors.

{
  "errors": [
    "Bad Request"
  ]
}

Internal Server Error

API error response.

Expand All

Field

Type

Description

errors [required]

[object]

A list of errors.

{
  "errors": [
    {
      "detail": "Missing required attribute in body",
      "meta": {},
      "source": {
        "header": "Authorization",
        "parameter": "limit",
        "pointer": "/data/attributes/title"
      },
      "status": "400",
      "title": "Bad Request"
    }
  ]
}

Code Example

                  ## default
# 

# Path parameters
export project_id="a33671aa-24fd-4dcd-9b33-a8ec7dde7751"
export dataset_id="9f64e5c7-dc5a-45c8-a17c-1b85f0bec97d"
# Curl command
curl -X POST "https://api.ap1.datadoghq.com"https://api.ap2.datadoghq.com"https://api.datadoghq.eu"https://api.ddog-gov.com"https://api.us2.ddog-gov.com"https://api.uk1.datadoghq.com"https://api.datadoghq.com"https://api.us3.datadoghq.com"https://api.us5.datadoghq.com/api/v2/llm-obs/v2/${project_id}/datasets/${dataset_id}/records/upload" \
-H "Accept: application/json" \
-H "Content-Type: multipart/form-data" \
-H "DD-API-KEY: ${DD_API_KEY}" \
-H "DD-APPLICATION-KEY: ${DD_APP_KEY}" \
-F file=@string

                

"""
Upload records to an Agent Observability dataset returns "OK" response
"""

from datadog_api_client import ApiClient, Configuration
from datadog_api_client.v2.api.agent_observability_api import AgentObservabilityApi

configuration = Configuration()
configuration.unstable_operations["upload_llm_obs_dataset_records_file"] = True
with ApiClient(configuration) as api_client:
    api_instance = AgentObservabilityApi(api_client)
    api_instance.upload_llm_obs_dataset_records_file(
        project_id="project_id",
        dataset_id="dataset_id",
    )

Instructions

First install the library and its dependencies and then save the example to example.py and run following commands:

    DD_SITE="datadoghq.comus3.datadoghq.comus5.datadoghq.comdatadoghq.euap1.datadoghq.comap2.datadoghq.comuk1.datadoghq.comddog-gov.comus2.ddog-gov.com" DD_API_KEY="<DD_API_KEY>" DD_APP_KEY="<DD_APP_KEY>" python3 "example.py"

# Upload records to an Agent Observability dataset returns "OK" response

require "datadog_api_client"
DatadogAPIClient.configure do |config|
  config.unstable_operations["v2.upload_llm_obs_dataset_records_file".to_sym] = true
end
api_instance = DatadogAPIClient::V2::AgentObservabilityAPI.new
p api_instance.upload_llm_obs_dataset_records_file("project_id", "dataset_id")

Instructions

First install the library and its dependencies and then save the example to example.rb and run following commands:

    DD_SITE="datadoghq.comus3.datadoghq.comus5.datadoghq.comdatadoghq.euap1.datadoghq.comap2.datadoghq.comuk1.datadoghq.comddog-gov.comus2.ddog-gov.com" DD_API_KEY="<DD_API_KEY>" DD_APP_KEY="<DD_APP_KEY>" rb "example.rb"

// Upload records to an Agent Observability dataset returns "OK" response

package main

import (
	"context"
	"fmt"
	"os"

	"github.com/DataDog/datadog-api-client-go/v2/api/datadog"
	"github.com/DataDog/datadog-api-client-go/v2/api/datadogV2"
)

func main() {
	ctx := datadog.NewDefaultContext(context.Background())
	configuration := datadog.NewConfiguration()
	configuration.SetUnstableOperationEnabled("v2.UploadLLMObsDatasetRecordsFile", true)
	apiClient := datadog.NewAPIClient(configuration)
	api := datadogV2.NewAgentObservabilityApi(apiClient)
	r, err := api.UploadLLMObsDatasetRecordsFile(ctx, "project_id", "dataset_id", *datadogV2.NewUploadLLMObsDatasetRecordsFileOptionalParameters())

	if err != nil {
		fmt.Fprintf(os.Stderr, "Error when calling `AgentObservabilityApi.UploadLLMObsDatasetRecordsFile`: %v\n", err)
		fmt.Fprintf(os.Stderr, "Full HTTP response: %v\n", r)
	}
}

Instructions

First install the library and its dependencies and then save the example to main.go and run following commands:

    DD_SITE="datadoghq.comus3.datadoghq.comus5.datadoghq.comdatadoghq.euap1.datadoghq.comap2.datadoghq.comuk1.datadoghq.comddog-gov.comus2.ddog-gov.com" DD_API_KEY="<DD_API_KEY>" DD_APP_KEY="<DD_APP_KEY>" go run "main.go"

// Upload records to an Agent Observability dataset returns "OK" response

import com.datadog.api.client.ApiClient;
import com.datadog.api.client.ApiException;
import com.datadog.api.client.v2.api.AgentObservabilityApi;

public class Example {
  public static void main(String[] args) {
    ApiClient defaultClient = ApiClient.getDefaultApiClient();
    defaultClient.setUnstableOperationEnabled("v2.uploadLLMObsDatasetRecordsFile", true);
    AgentObservabilityApi apiInstance = new AgentObservabilityApi(defaultClient);

    try {
      apiInstance.uploadLLMObsDatasetRecordsFile(
          "a33671aa-24fd-4dcd-9b33-a8ec7dde7751", "9f64e5c7-dc5a-45c8-a17c-1b85f0bec97d");
    } catch (ApiException e) {
      System.err.println(
          "Exception when calling AgentObservabilityApi#uploadLLMObsDatasetRecordsFile");
      System.err.println("Status code: " + e.getCode());
      System.err.println("Reason: " + e.getResponseBody());
      System.err.println("Response headers: " + e.getResponseHeaders());
      e.printStackTrace();
    }
  }
}

Instructions

First install the library and its dependencies and then save the example to Example.java and run following commands:

    DD_SITE="datadoghq.comus3.datadoghq.comus5.datadoghq.comdatadoghq.euap1.datadoghq.comap2.datadoghq.comuk1.datadoghq.comddog-gov.comus2.ddog-gov.com" DD_API_KEY="<DD_API_KEY>" DD_APP_KEY="<DD_APP_KEY>" java "Example.java"

// Upload records to an Agent Observability dataset returns "OK" response
use datadog_api_client::datadog;
use datadog_api_client::datadogV2::api_agent_observability::AgentObservabilityAPI;
use datadog_api_client::datadogV2::api_agent_observability::UploadLLMObsDatasetRecordsFileOptionalParams;

#[tokio::main]
async fn main() {
    let mut configuration = datadog::Configuration::new();
    configuration.set_unstable_operation_enabled("v2.UploadLLMObsDatasetRecordsFile", true);
    let api = AgentObservabilityAPI::with_config(configuration);
    let resp = api
        .upload_llm_obs_dataset_records_file(
            "project_id".to_string(),
            "dataset_id".to_string(),
            UploadLLMObsDatasetRecordsFileOptionalParams::default(),
        )
        .await;
    if let Ok(value) = resp {
        println!("{:#?}", value);
    } else {
        println!("{:#?}", resp.unwrap_err());
    }
}

Instructions

First install the library and its dependencies and then save the example to src/main.rs and run following commands:

    DD_SITE="datadoghq.comus3.datadoghq.comus5.datadoghq.comdatadoghq.euap1.datadoghq.comap2.datadoghq.comuk1.datadoghq.comddog-gov.comus2.ddog-gov.com" DD_API_KEY="<DD_API_KEY>" DD_APP_KEY="<DD_APP_KEY>" cargo run

/**
 * Upload records to an Agent Observability dataset returns "OK" response
 */

import { client, v2 } from "@datadog/datadog-api-client";

const configuration = client.createConfiguration();
configuration.unstableOperations["v2.uploadLLMObsDatasetRecordsFile"] = true;
const apiInstance = new v2.AgentObservabilityApi(configuration);

const params: v2.AgentObservabilityApiUploadLLMObsDatasetRecordsFileRequest = {
  projectId: "project_id",
  datasetId: "dataset_id",
};

apiInstance
  .uploadLLMObsDatasetRecordsFile(params)
  .then((data: any) => {
    console.log(
      "API called successfully. Returned data: " + JSON.stringify(data)
    );
  })
  .catch((error: any) => console.error(error));

Instructions

First install the library and its dependencies and then save the example to example.ts and run following commands:

    DD_SITE="datadoghq.comus3.datadoghq.comus5.datadoghq.comdatadoghq.euap1.datadoghq.comap2.datadoghq.comuk1.datadoghq.comddog-gov.comus2.ddog-gov.com" DD_API_KEY="<DD_API_KEY>" DD_APP_KEY="<DD_APP_KEY>" tsc "example.ts"

Upload records to an Agent Observability dataset

Overview

Arguments

Path Parameters

Query Strings

Request

Body Data (required)

Response

Code Example

Upload records to an Agent Observability dataset

Upload records to an Agent Observability dataset

Instructions

Upload records to an Agent Observability dataset

Instructions

Upload records to an Agent Observability dataset

Instructions

Upload records to an Agent Observability dataset

Instructions

Upload records to an Agent Observability dataset

Instructions

Upload records to an Agent Observability dataset

Instructions

How can I help you today?