Upload records to an LLM Observability dataset

Note: This endpoint is in preview and is subject to change. If you have any feedback, contact Datadog support.

POST https://api.ap1.datadoghq.com/api/v2/llm-obs/v2/{project_id}/datasets/{dataset_id}/records/uploadhttps://api.ap2.datadoghq.com/api/v2/llm-obs/v2/{project_id}/datasets/{dataset_id}/records/uploadhttps://api.datadoghq.eu/api/v2/llm-obs/v2/{project_id}/datasets/{dataset_id}/records/uploadhttps://api.ddog-gov.com/api/v2/llm-obs/v2/{project_id}/datasets/{dataset_id}/records/uploadhttps://api.us2.ddog-gov.com/api/v2/llm-obs/v2/{project_id}/datasets/{dataset_id}/records/uploadhttps://api.datadoghq.com/api/v2/llm-obs/v2/{project_id}/datasets/{dataset_id}/records/uploadhttps://api.us3.datadoghq.com/api/v2/llm-obs/v2/{project_id}/datasets/{dataset_id}/records/uploadhttps://api.us5.datadoghq.com/api/v2/llm-obs/v2/{project_id}/datasets/{dataset_id}/records/upload

Overview

Upload records to a dataset from a file. The request is a multipart/form-data upload containing a single file part. Currently only CSV is supported. The CSV must include an input column. Optional columns are id, expected_output, metadata, and tags.

The response is a Server-Sent Events stream (text/event-stream) emitting progress updates while records are processed. The stream emits the following named events:

  • progress: incremental record counts written so far.
  • completed: terminal event with a JSON body containing records_created.
  • error: terminal event with a JSON body containing an error message.

Arguments

Path Parameters

Name

Type

Description

project_id [required]

string

The ID of the LLM Observability project.

dataset_id [required]

string

The ID of the LLM Observability dataset.

Query Strings

Name

Type

Description

deduplicate

boolean

Whether to skip records whose input already exists in the dataset. Defaults to false.

overwrite

boolean

Whether to overwrite existing records that share the same user-provided id. Defaults to true.

tags

array

Tags to apply to every uploaded record, in addition to any tags defined on individual rows. Can be repeated, e.g. tags=env:prod&tags=team:ai.

include[user_data]

boolean

Whether to enrich the response with user metadata.

Request

Body Data (required)

Multipart upload payload containing the records file.

Expand All

Field

Type

Description

file

binary

The records file to upload. Currently only CSV is supported. The file must include an input column. Optional columns include id, expected_output, metadata, and tags.

{
  "file": "string"
}

Response

OK

Bad Request

API error response.

Expand All

Field

Type

Description

errors [required]

[object]

A list of errors.

detail

string

A human-readable explanation specific to this occurrence of the error.

meta

object

Non-standard meta-information about the error

source

object

References to the source of the error.

header

string

A string indicating the name of a single request header which caused the error.

parameter

string

A string indicating which URI query parameter caused the error.

pointer

string

A JSON pointer to the value in the request document that caused the error.

status

string

Status code of the response.

title

string

Short human-readable summary of the error.

{
  "errors": [
    {
      "detail": "Missing required attribute in body",
      "meta": {},
      "source": {
        "header": "Authorization",
        "parameter": "limit",
        "pointer": "/data/attributes/title"
      },
      "status": "400",
      "title": "Bad Request"
    }
  ]
}

Unauthorized

API error response.

Expand All

Field

Type

Description

errors [required]

[object]

A list of errors.

detail

string

A human-readable explanation specific to this occurrence of the error.

meta

object

Non-standard meta-information about the error

source

object

References to the source of the error.

header

string

A string indicating the name of a single request header which caused the error.

parameter

string

A string indicating which URI query parameter caused the error.

pointer

string

A JSON pointer to the value in the request document that caused the error.

status

string

Status code of the response.

title

string

Short human-readable summary of the error.

{
  "errors": [
    {
      "detail": "Missing required attribute in body",
      "meta": {},
      "source": {
        "header": "Authorization",
        "parameter": "limit",
        "pointer": "/data/attributes/title"
      },
      "status": "400",
      "title": "Bad Request"
    }
  ]
}

Forbidden

API error response.

Expand All

Field

Type

Description

errors [required]

[object]

A list of errors.

detail

string

A human-readable explanation specific to this occurrence of the error.

meta

object

Non-standard meta-information about the error

source

object

References to the source of the error.

header

string

A string indicating the name of a single request header which caused the error.

parameter

string

A string indicating which URI query parameter caused the error.

pointer

string

A JSON pointer to the value in the request document that caused the error.

status

string

Status code of the response.

title

string

Short human-readable summary of the error.

{
  "errors": [
    {
      "detail": "Missing required attribute in body",
      "meta": {},
      "source": {
        "header": "Authorization",
        "parameter": "limit",
        "pointer": "/data/attributes/title"
      },
      "status": "400",
      "title": "Bad Request"
    }
  ]
}

Not Found

API error response.

Expand All

Field

Type

Description

errors [required]

[object]

A list of errors.

detail

string

A human-readable explanation specific to this occurrence of the error.

meta

object

Non-standard meta-information about the error

source

object

References to the source of the error.

header

string

A string indicating the name of a single request header which caused the error.

parameter

string

A string indicating which URI query parameter caused the error.

pointer

string

A JSON pointer to the value in the request document that caused the error.

status

string

Status code of the response.

title

string

Short human-readable summary of the error.

{
  "errors": [
    {
      "detail": "Missing required attribute in body",
      "meta": {},
      "source": {
        "header": "Authorization",
        "parameter": "limit",
        "pointer": "/data/attributes/title"
      },
      "status": "400",
      "title": "Bad Request"
    }
  ]
}

Too many requests

API error response.

Expand All

Field

Type

Description

errors [required]

[string]

A list of errors.

{
  "errors": [
    "Bad Request"
  ]
}

Internal Server Error

API error response.

Expand All

Field

Type

Description

errors [required]

[object]

A list of errors.

detail

string

A human-readable explanation specific to this occurrence of the error.

meta

object

Non-standard meta-information about the error

source

object

References to the source of the error.

header

string

A string indicating the name of a single request header which caused the error.

parameter

string

A string indicating which URI query parameter caused the error.

pointer

string

A JSON pointer to the value in the request document that caused the error.

status

string

Status code of the response.

title

string

Short human-readable summary of the error.

{
  "errors": [
    {
      "detail": "Missing required attribute in body",
      "meta": {},
      "source": {
        "header": "Authorization",
        "parameter": "limit",
        "pointer": "/data/attributes/title"
      },
      "status": "400",
      "title": "Bad Request"
    }
  ]
}

Code Example

                  ## default
# 

# Path parameters
export project_id="a33671aa-24fd-4dcd-9b33-a8ec7dde7751"
export dataset_id="9f64e5c7-dc5a-45c8-a17c-1b85f0bec97d"
# Curl command
curl -X POST "https://api.ap1.datadoghq.com"https://api.ap2.datadoghq.com"https://api.datadoghq.eu"https://api.ddog-gov.com"https://api.us2.ddog-gov.com"https://api.datadoghq.com"https://api.us3.datadoghq.com"https://api.us5.datadoghq.com/api/v2/llm-obs/v2/${project_id}/datasets/${dataset_id}/records/upload" \ -H "Accept: application/json" \ -H "Content-Type: multipart/form-data" \ -H "DD-API-KEY: ${DD_API_KEY}" \ -H "DD-APPLICATION-KEY: ${DD_APP_KEY}" \ -F file=@string