--- title: Push events for an LLM Observability experiment description: Datadog, the leading service for cloud-scale monitoring. breadcrumbs: Docs > API Reference > LLM Observability --- # Push events for an LLM Observability experiment{% #push-events-for-an-llm-observability-experiment %} Copy pageCopied {% tab title="v2" %} **Note**: This endpoint is in preview and is subject to change. If you have any feedback, contact [Datadog support](https://docs.datadoghq.com/help/). | Datadog site | API endpoint | | ----------------- | --------------------------------------------------------------------------------------- | | ap1.datadoghq.com | POST https://api.ap1.datadoghq.com/api/v2/llm-obs/v1/experiments/{experiment_id}/events | | ap2.datadoghq.com | POST https://api.ap2.datadoghq.com/api/v2/llm-obs/v1/experiments/{experiment_id}/events | | app.datadoghq.eu | POST https://api.datadoghq.eu/api/v2/llm-obs/v1/experiments/{experiment_id}/events | | app.ddog-gov.com | POST https://api.ddog-gov.com/api/v2/llm-obs/v1/experiments/{experiment_id}/events | | us2.ddog-gov.com | POST https://api.us2.ddog-gov.com/api/v2/llm-obs/v1/experiments/{experiment_id}/events | | app.datadoghq.com | POST https://api.datadoghq.com/api/v2/llm-obs/v1/experiments/{experiment_id}/events | | us3.datadoghq.com | POST https://api.us3.datadoghq.com/api/v2/llm-obs/v1/experiments/{experiment_id}/events | | us5.datadoghq.com | POST https://api.us5.datadoghq.com/api/v2/llm-obs/v1/experiments/{experiment_id}/events | ### Overview Push spans and metrics for an LLM Observability experiment. ### Arguments #### Path Parameters | Name | Type | Description | | ------------------------------- | ------ | ------------------------------------------- | | experiment_id [*required*] | string | The ID of the LLM Observability experiment. | ### Request #### Body Data (required) Experiment events payload. {% tab title="Model" %} | Parent field | Field | Type | Description | | ------------ | ------------------------------ | ------------------- | ------------------------------------------------------------------------------------------------------------------ | | | data [*required*] | object | Data object for pushing experiment events. | | data | attributes [*required*] | object | Attributes for pushing experiment events including spans and metrics. | | attributes | metrics | [object] | List of metrics to push for the experiment. | | metrics | assessment | enum | Assessment result for an LLM Observability experiment metric. Allowed enum values: `pass,fail` | | metrics | boolean_value | boolean | Boolean value. Used when `metric_type` is `boolean`. | | metrics | categorical_value | string | Categorical value. Used when `metric_type` is `categorical`. | | metrics | error | object | Error details for an experiment metric evaluation. | | error | message | string | Error message associated with the metric evaluation. | | metrics | json_value | object | JSON value. Used when `metric_type` is `json`. | | metrics | label [*required*] | string | Label or name for the metric. | | metrics | metadata | object | Arbitrary metadata associated with the metric. | | metrics | metric_type [*required*] | enum | Type of metric recorded for an LLM Observability experiment. Allowed enum values: `score,categorical,boolean,json` | | metrics | reasoning | string | Human-readable reasoning for the metric value. | | metrics | score_value | double | Numeric score value. Used when `metric_type` is `score`. | | metrics | span_id [*required*] | string | The ID of the span this metric measures. | | metrics | tags | [string] | List of tags associated with the metric. | | metrics | timestamp_ms [*required*] | int64 | Timestamp when the metric was recorded, in milliseconds since Unix epoch. | | attributes | spans | [object] | List of spans to push for the experiment. | | spans | dataset_id [*required*] | string | Dataset ID associated with this span. | | spans | duration [*required*] | int64 | Duration of the span in nanoseconds. | | spans | meta | object | Metadata associated with an experiment span. | | meta | error | object | Error details for an experiment span. | | error | message | string | Error message. | | error | stack | string | Stack trace of the error. | | error | type | string | The error type or exception class name. | | meta | expected_output | object | Expected output for the span, used for evaluation. | | meta | input | object | Represents any valid JSON value. | | input | Option 1 | string | A scalar string value. | | input | Option 2 | double | A scalar numeric value. | | input | Option 3 | object | An arbitrary object value with additional properties. | | input | Option 4 | [ ] | An array of arbitrary values. | | Option 4 | Option 1 | string | A scalar string value. | | Option 4 | Option 2 | double | A scalar numeric value. | | Option 4 | Option 3 | object | An arbitrary object value with additional properties. | | Option 4 | Option 4 | boolean | A scalar boolean value. | | input | Option 5 | boolean | A scalar boolean value. | | meta | output | object | Represents any valid JSON value. | | output | Option 1 | string | A scalar string value. | | output | Option 2 | double | A scalar numeric value. | | output | Option 3 | object | An arbitrary object value with additional properties. | | output | Option 4 | [ ] | An array of arbitrary values. | | Option 4 | Option 1 | string | A scalar string value. | | Option 4 | Option 2 | double | A scalar numeric value. | | Option 4 | Option 3 | object | An arbitrary object value with additional properties. | | Option 4 | Option 4 | boolean | A scalar boolean value. | | output | Option 5 | boolean | A scalar boolean value. | | spans | name [*required*] | string | Name of the span. | | spans | project_id [*required*] | string | Project ID associated with this span. | | spans | span_id [*required*] | string | Unique identifier of the span. | | spans | start_ns [*required*] | int64 | Start time of the span in nanoseconds since Unix epoch. | | spans | status [*required*] | enum | Status of the span. Allowed enum values: `ok,error` | | spans | tags | [string] | List of tags associated with the span. | | spans | trace_id [*required*] | string | Trace ID for the span. | | data | type [*required*] | enum | Resource type for LLM Observability experiment events. Allowed enum values: `events` | {% /tab %} {% tab title="Example" %} ```json { "data": { "attributes": { "metrics": [ { "assessment": "pass", "boolean_value": false, "categorical_value": "string", "error": { "message": "string" }, "json_value": {}, "label": "faithfulness", "metadata": {}, "metric_type": "score", "reasoning": "string", "score_value": "number", "span_id": "span-7a1b2c3d", "tags": [], "timestamp_ms": 1705314600000 } ], "spans": [ { "dataset_id": "9f64e5c7-dc5a-45c8-a17c-1b85f0bec97d", "duration": 1500000000, "meta": { "error": { "message": "Model response timed out", "stack": "Traceback (most recent call last):\n File \"main.py\", line 10, in \n response = model.generate(input)\n File \"model.py\", line 45, in generate\n raise TimeoutError(\"Model response timed out\")\nTimeoutError: Model response timed out", "type": "TimeoutError" }, "expected_output": {}, "input": { "description": "undefined", "type": "undefined" }, "output": { "description": "undefined", "type": "undefined" } }, "name": "llm_call", "project_id": "a33671aa-24fd-4dcd-9b33-a8ec7dde7751", "span_id": "span-7a1b2c3d", "start_ns": 1705314600000000000, "status": "ok", "tags": [], "trace_id": "abc123def456" } ] }, "type": "events" } } ``` {% /tab %} ### Response {% tab title="202" %} Accepted {% /tab %} {% tab title="400" %} Bad Request {% tab title="Model" %} API error response. | Parent field | Field | Type | Description | | ------------ | ------------------------ | -------- | ------------------------------------------------------------------------------- | | | errors [*required*] | [object] | A list of errors. | | errors | detail | string | A human-readable explanation specific to this occurrence of the error. | | errors | meta | object | Non-standard meta-information about the error | | errors | source | object | References to the source of the error. | | source | header | string | A string indicating the name of a single request header which caused the error. | | source | parameter | string | A string indicating which URI query parameter caused the error. | | source | pointer | string | A JSON pointer to the value in the request document that caused the error. | | errors | status | string | Status code of the response. | | errors | title | string | Short human-readable summary of the error. | {% /tab %} {% tab title="Example" %} ```json { "errors": [ { "detail": "Missing required attribute in body", "meta": {}, "source": { "header": "Authorization", "parameter": "limit", "pointer": "/data/attributes/title" }, "status": "400", "title": "Bad Request" } ] } ``` {% /tab %} {% /tab %} {% tab title="401" %} Unauthorized {% tab title="Model" %} API error response. | Parent field | Field | Type | Description | | ------------ | ------------------------ | -------- | ------------------------------------------------------------------------------- | | | errors [*required*] | [object] | A list of errors. | | errors | detail | string | A human-readable explanation specific to this occurrence of the error. | | errors | meta | object | Non-standard meta-information about the error | | errors | source | object | References to the source of the error. | | source | header | string | A string indicating the name of a single request header which caused the error. | | source | parameter | string | A string indicating which URI query parameter caused the error. | | source | pointer | string | A JSON pointer to the value in the request document that caused the error. | | errors | status | string | Status code of the response. | | errors | title | string | Short human-readable summary of the error. | {% /tab %} {% tab title="Example" %} ```json { "errors": [ { "detail": "Missing required attribute in body", "meta": {}, "source": { "header": "Authorization", "parameter": "limit", "pointer": "/data/attributes/title" }, "status": "400", "title": "Bad Request" } ] } ``` {% /tab %} {% /tab %} {% tab title="403" %} Forbidden {% tab title="Model" %} API error response. | Parent field | Field | Type | Description | | ------------ | ------------------------ | -------- | ------------------------------------------------------------------------------- | | | errors [*required*] | [object] | A list of errors. | | errors | detail | string | A human-readable explanation specific to this occurrence of the error. | | errors | meta | object | Non-standard meta-information about the error | | errors | source | object | References to the source of the error. | | source | header | string | A string indicating the name of a single request header which caused the error. | | source | parameter | string | A string indicating which URI query parameter caused the error. | | source | pointer | string | A JSON pointer to the value in the request document that caused the error. | | errors | status | string | Status code of the response. | | errors | title | string | Short human-readable summary of the error. | {% /tab %} {% tab title="Example" %} ```json { "errors": [ { "detail": "Missing required attribute in body", "meta": {}, "source": { "header": "Authorization", "parameter": "limit", "pointer": "/data/attributes/title" }, "status": "400", "title": "Bad Request" } ] } ``` {% /tab %} {% /tab %} {% tab title="404" %} Not Found {% tab title="Model" %} API error response. | Parent field | Field | Type | Description | | ------------ | ------------------------ | -------- | ------------------------------------------------------------------------------- | | | errors [*required*] | [object] | A list of errors. | | errors | detail | string | A human-readable explanation specific to this occurrence of the error. | | errors | meta | object | Non-standard meta-information about the error | | errors | source | object | References to the source of the error. | | source | header | string | A string indicating the name of a single request header which caused the error. | | source | parameter | string | A string indicating which URI query parameter caused the error. | | source | pointer | string | A JSON pointer to the value in the request document that caused the error. | | errors | status | string | Status code of the response. | | errors | title | string | Short human-readable summary of the error. | {% /tab %} {% tab title="Example" %} ```json { "errors": [ { "detail": "Missing required attribute in body", "meta": {}, "source": { "header": "Authorization", "parameter": "limit", "pointer": "/data/attributes/title" }, "status": "400", "title": "Bad Request" } ] } ``` {% /tab %} {% /tab %} {% tab title="429" %} Too many requests {% tab title="Model" %} API error response. | Field | Type | Description | | ------------------------ | -------- | ----------------- | | errors [*required*] | [string] | A list of errors. | {% /tab %} {% tab title="Example" %} ```json { "errors": [ "Bad Request" ] } ``` {% /tab %} {% /tab %} ### Code Example ##### \## default # \# Path parameters export experiment_id="3fd6b5e0-8910-4b1c-a7d0-5b84de329012" \# Curl command curl -X POST "https://api.datadoghq.com/api/v2/llm-obs/v1/experiments/${experiment_id}/events" \ -H "Accept: application/json" \ -H "Content-Type: application/json" \ -H "DD-API-KEY: ${DD_API_KEY}" \ -H "DD-APPLICATION-KEY: ${DD_APP_KEY}" \ -d @- << EOF { "data": { "attributes": { "metrics": [ { "assessment": "pass", "label": "faithfulness", "metric_type": "score", "span_id": "span-7a1b2c3d", "timestamp_ms": 1705314600000 } ], "spans": [ { "dataset_id": "9f64e5c7-dc5a-45c8-a17c-1b85f0bec97d", "duration": 1500000000, "name": "llm_call", "project_id": "a33671aa-24fd-4dcd-9b33-a8ec7dde7751", "span_id": "span-7a1b2c3d", "start_ns": 1705314600000000000, "status": "ok", "trace_id": "abc123def456" } ] }, "type": "events" } } EOF ##### ```python """ Push events for an LLM Observability experiment returns "Accepted" response """ from datadog_api_client import ApiClient, Configuration from datadog_api_client.v2.api.llm_observability_api import LLMObservabilityApi from datadog_api_client.v2.model.llm_obs_event_type import LLMObsEventType from datadog_api_client.v2.model.llm_obs_experiment_events_data_attributes_request import ( LLMObsExperimentEventsDataAttributesRequest, ) from datadog_api_client.v2.model.llm_obs_experiment_events_data_request import LLMObsExperimentEventsDataRequest from datadog_api_client.v2.model.llm_obs_experiment_events_request import LLMObsExperimentEventsRequest from datadog_api_client.v2.model.llm_obs_experiment_metric import LLMObsExperimentMetric from datadog_api_client.v2.model.llm_obs_experiment_metric_error import LLMObsExperimentMetricError from datadog_api_client.v2.model.llm_obs_experiment_span import LLMObsExperimentSpan from datadog_api_client.v2.model.llm_obs_experiment_span_error import LLMObsExperimentSpanError from datadog_api_client.v2.model.llm_obs_experiment_span_meta import LLMObsExperimentSpanMeta from datadog_api_client.v2.model.llm_obs_experiment_span_status import LLMObsExperimentSpanStatus from datadog_api_client.v2.model.llm_obs_metric_assessment import LLMObsMetricAssessment from datadog_api_client.v2.model.llm_obs_metric_score_type import LLMObsMetricScoreType body = LLMObsExperimentEventsRequest( data=LLMObsExperimentEventsDataRequest( attributes=LLMObsExperimentEventsDataAttributesRequest( metrics=[ LLMObsExperimentMetric( assessment=LLMObsMetricAssessment.PASS, error=LLMObsExperimentMetricError(), label="faithfulness", metric_type=LLMObsMetricScoreType.SCORE, span_id="span-7a1b2c3d", tags=[], timestamp_ms=1705314600000, ), ], spans=[ LLMObsExperimentSpan( dataset_id="9f64e5c7-dc5a-45c8-a17c-1b85f0bec97d", duration=1500000000, meta=LLMObsExperimentSpanMeta( error=LLMObsExperimentSpanError( message="Model response timed out", stack='Traceback (most recent call last):\n File "main.py", line 10, in \n response = model.generate(input)\n File "model.py", line 45, in generate\n raise TimeoutError("Model response timed out")\nTimeoutError: Model response timed out', type="TimeoutError", ), input=None, output=None, ), name="llm_call", project_id="a33671aa-24fd-4dcd-9b33-a8ec7dde7751", span_id="span-7a1b2c3d", start_ns=1705314600000000000, status=LLMObsExperimentSpanStatus.OK, tags=[], trace_id="abc123def456", ), ], ), type=LLMObsEventType.EVENTS, ), ) configuration = Configuration() configuration.unstable_operations["create_llm_obs_experiment_events"] = True with ApiClient(configuration) as api_client: api_instance = LLMObservabilityApi(api_client) api_instance.create_llm_obs_experiment_events(experiment_id="experiment_id", body=body) ``` #### Instructions First [install the library and its dependencies](https://docs.datadoghq.com/api/latest.md?code-lang=python) and then save the example to `example.py` and run following commands: DD_SITE="datadoghq.com" DD_API_KEY="" DD_APP_KEY="" python3 "example.py" ##### ```ruby # Push events for an LLM Observability experiment returns "Accepted" response require "datadog_api_client" DatadogAPIClient.configure do |config| config.unstable_operations["v2.create_llm_obs_experiment_events".to_sym] = true end api_instance = DatadogAPIClient::V2::LLMObservabilityAPI.new body = DatadogAPIClient::V2::LLMObsExperimentEventsRequest.new({ data: DatadogAPIClient::V2::LLMObsExperimentEventsDataRequest.new({ attributes: DatadogAPIClient::V2::LLMObsExperimentEventsDataAttributesRequest.new({ metrics: [ DatadogAPIClient::V2::LLMObsExperimentMetric.new({ assessment: DatadogAPIClient::V2::LLMObsMetricAssessment::PASS, error: DatadogAPIClient::V2::LLMObsExperimentMetricError.new({}), label: "faithfulness", metric_type: DatadogAPIClient::V2::LLMObsMetricScoreType::SCORE, span_id: "span-7a1b2c3d", tags: [], timestamp_ms: 1705314600000, }), ], spans: [ DatadogAPIClient::V2::LLMObsExperimentSpan.new({ dataset_id: "9f64e5c7-dc5a-45c8-a17c-1b85f0bec97d", duration: 1500000000, meta: DatadogAPIClient::V2::LLMObsExperimentSpanMeta.new({ error: DatadogAPIClient::V2::LLMObsExperimentSpanError.new({ message: "Model response timed out", stack: 'Traceback (most recent call last):\n File "main.py", line 10, in \n response = model.generate(input)\n File "model.py", line 45, in generate\n raise TimeoutError("Model response timed out")\nTimeoutError: Model response timed out', type: "TimeoutError", }), input: nil, output: nil, }), name: "llm_call", project_id: "a33671aa-24fd-4dcd-9b33-a8ec7dde7751", span_id: "span-7a1b2c3d", start_ns: 1705314600000000000, status: DatadogAPIClient::V2::LLMObsExperimentSpanStatus::OK, tags: [], trace_id: "abc123def456", }), ], }), type: DatadogAPIClient::V2::LLMObsEventType::EVENTS, }), }) p api_instance.create_llm_obs_experiment_events("experiment_id", body) ``` #### Instructions First [install the library and its dependencies](https://docs.datadoghq.com/api/latest.md?code-lang=ruby) and then save the example to `example.rb` and run following commands: DD_SITE="datadoghq.com" DD_API_KEY="" DD_APP_KEY="" rb "example.rb" ##### ```go // Push events for an LLM Observability experiment returns "Accepted" response package main import ( "context" "fmt" "os" "github.com/DataDog/datadog-api-client-go/v2/api/datadog" "github.com/DataDog/datadog-api-client-go/v2/api/datadogV2" ) func main() { body := datadogV2.LLMObsExperimentEventsRequest{ Data: datadogV2.LLMObsExperimentEventsDataRequest{ Attributes: datadogV2.LLMObsExperimentEventsDataAttributesRequest{ Metrics: []datadogV2.LLMObsExperimentMetric{ { Assessment: datadogV2.LLMOBSMETRICASSESSMENT_PASS.Ptr(), Error: &datadogV2.LLMObsExperimentMetricError{}, Label: "faithfulness", MetricType: datadogV2.LLMOBSMETRICSCORETYPE_SCORE, SpanId: "span-7a1b2c3d", Tags: []string{}, TimestampMs: 1705314600000, }, }, Spans: []datadogV2.LLMObsExperimentSpan{ { DatasetId: "9f64e5c7-dc5a-45c8-a17c-1b85f0bec97d", Duration: 1500000000, Meta: &datadogV2.LLMObsExperimentSpanMeta{ Error: &datadogV2.LLMObsExperimentSpanError{ Message: datadog.PtrString("Model response timed out"), Stack: datadog.PtrString(`Traceback (most recent call last): File "main.py", line 10, in response = model.generate(input) File "model.py", line 45, in generate raise TimeoutError("Model response timed out") TimeoutError: Model response timed out`), Type: datadog.PtrString("TimeoutError"), }, Input: *datadogV2.NewNullableAnyValue(nil), Output: *datadogV2.NewNullableAnyValue(nil), }, Name: "llm_call", ProjectId: "a33671aa-24fd-4dcd-9b33-a8ec7dde7751", SpanId: "span-7a1b2c3d", StartNs: 1705314600000000000, Status: datadogV2.LLMOBSEXPERIMENTSPANSTATUS_OK, Tags: []string{}, TraceId: "abc123def456", }, }, }, Type: datadogV2.LLMOBSEVENTTYPE_EVENTS, }, } ctx := datadog.NewDefaultContext(context.Background()) configuration := datadog.NewConfiguration() configuration.SetUnstableOperationEnabled("v2.CreateLLMObsExperimentEvents", true) apiClient := datadog.NewAPIClient(configuration) api := datadogV2.NewLLMObservabilityApi(apiClient) r, err := api.CreateLLMObsExperimentEvents(ctx, "experiment_id", body) if err != nil { fmt.Fprintf(os.Stderr, "Error when calling `LLMObservabilityApi.CreateLLMObsExperimentEvents`: %v\n", err) fmt.Fprintf(os.Stderr, "Full HTTP response: %v\n", r) } } ``` #### Instructions First [install the library and its dependencies](https://docs.datadoghq.com/api/latest.md?code-lang=go) and then save the example to `main.go` and run following commands: DD_SITE="datadoghq.com" DD_API_KEY="" DD_APP_KEY="" go run "main.go" ##### ```java // Push events for an LLM Observability experiment returns "Accepted" response import com.datadog.api.client.ApiClient; import com.datadog.api.client.ApiException; import com.datadog.api.client.v2.api.LlmObservabilityApi; import com.datadog.api.client.v2.model.LLMObsEventType; import com.datadog.api.client.v2.model.LLMObsExperimentEventsDataAttributesRequest; import com.datadog.api.client.v2.model.LLMObsExperimentEventsDataRequest; import com.datadog.api.client.v2.model.LLMObsExperimentEventsRequest; import com.datadog.api.client.v2.model.LLMObsExperimentMetric; import com.datadog.api.client.v2.model.LLMObsExperimentMetricError; import com.datadog.api.client.v2.model.LLMObsExperimentSpan; import com.datadog.api.client.v2.model.LLMObsExperimentSpanError; import com.datadog.api.client.v2.model.LLMObsExperimentSpanMeta; import com.datadog.api.client.v2.model.LLMObsExperimentSpanStatus; import com.datadog.api.client.v2.model.LLMObsMetricAssessment; import com.datadog.api.client.v2.model.LLMObsMetricScoreType; import java.util.Collections; public class Example { public static void main(String[] args) { ApiClient defaultClient = ApiClient.getDefaultApiClient(); defaultClient.setUnstableOperationEnabled("v2.createLLMObsExperimentEvents", true); LlmObservabilityApi apiInstance = new LlmObservabilityApi(defaultClient); LLMObsExperimentEventsRequest body = new LLMObsExperimentEventsRequest() .data( new LLMObsExperimentEventsDataRequest() .attributes( new LLMObsExperimentEventsDataAttributesRequest() .metrics( Collections.singletonList( new LLMObsExperimentMetric() .assessment(LLMObsMetricAssessment.PASS) .error(new LLMObsExperimentMetricError()) .label("faithfulness") .metricType(LLMObsMetricScoreType.SCORE) .spanId("span-7a1b2c3d") .timestampMs(1705314600000L))) .spans( Collections.singletonList( new LLMObsExperimentSpan() .datasetId("9f64e5c7-dc5a-45c8-a17c-1b85f0bec97d") .duration(1500000000L) .meta( new LLMObsExperimentSpanMeta() .error( new LLMObsExperimentSpanError() .message("Model response timed out") .stack( """ Traceback (most recent call last): File "main.py", line 10, in response = model.generate(input) File "model.py", line 45, in generate raise TimeoutError("Model response timed out") TimeoutError: Model response timed out """) .type("TimeoutError")) .input(null) .output(null)) .name("llm_call") .projectId("a33671aa-24fd-4dcd-9b33-a8ec7dde7751") .spanId("span-7a1b2c3d") .startNs(1705314600000000000L) .status(LLMObsExperimentSpanStatus.OK) .traceId("abc123def456")))) .type(LLMObsEventType.EVENTS)); try { apiInstance.createLLMObsExperimentEvents("3fd6b5e0-8910-4b1c-a7d0-5b84de329012", body); } catch (ApiException e) { System.err.println("Exception when calling LlmObservabilityApi#createLLMObsExperimentEvents"); System.err.println("Status code: " + e.getCode()); System.err.println("Reason: " + e.getResponseBody()); System.err.println("Response headers: " + e.getResponseHeaders()); e.printStackTrace(); } } } ``` #### Instructions First [install the library and its dependencies](https://docs.datadoghq.com/api/latest.md?code-lang=java) and then save the example to `Example.java` and run following commands: DD_SITE="datadoghq.com" DD_API_KEY="" DD_APP_KEY="" java "Example.java" ##### ```rust // Push events for an LLM Observability experiment returns "Accepted" response use datadog_api_client::datadog; use datadog_api_client::datadogV2::api_llm_observability::LLMObservabilityAPI; use datadog_api_client::datadogV2::model::LLMObsEventType; use datadog_api_client::datadogV2::model::LLMObsExperimentEventsDataAttributesRequest; use datadog_api_client::datadogV2::model::LLMObsExperimentEventsDataRequest; use datadog_api_client::datadogV2::model::LLMObsExperimentEventsRequest; use datadog_api_client::datadogV2::model::LLMObsExperimentMetric; use datadog_api_client::datadogV2::model::LLMObsExperimentMetricError; use datadog_api_client::datadogV2::model::LLMObsExperimentSpan; use datadog_api_client::datadogV2::model::LLMObsExperimentSpanError; use datadog_api_client::datadogV2::model::LLMObsExperimentSpanMeta; use datadog_api_client::datadogV2::model::LLMObsExperimentSpanStatus; use datadog_api_client::datadogV2::model::LLMObsMetricAssessment; use datadog_api_client::datadogV2::model::LLMObsMetricScoreType; #[tokio::main] async fn main() { let body = LLMObsExperimentEventsRequest::new(LLMObsExperimentEventsDataRequest::new( LLMObsExperimentEventsDataAttributesRequest::new() .metrics(vec![LLMObsExperimentMetric::new( "faithfulness".to_string(), LLMObsMetricScoreType::SCORE, "span-7a1b2c3d".to_string(), 1705314600000, ) .assessment(LLMObsMetricAssessment::PASS) .error(LLMObsExperimentMetricError::new()) .tags(vec![])]) .spans(vec![LLMObsExperimentSpan::new( "9f64e5c7-dc5a-45c8-a17c-1b85f0bec97d".to_string(), 1500000000, "llm_call".to_string(), "a33671aa-24fd-4dcd-9b33-a8ec7dde7751".to_string(), "span-7a1b2c3d".to_string(), 1705314600000000000, LLMObsExperimentSpanStatus::OK, "abc123def456".to_string(), ) .meta( LLMObsExperimentSpanMeta::new() .error( LLMObsExperimentSpanError::new() .message("Model response timed out".to_string()) .stack( r#"Traceback (most recent call last): File "main.py", line 10, in response = model.generate(input) File "model.py", line 45, in generate raise TimeoutError("Model response timed out") TimeoutError: Model response timed out"# .to_string(), ) .type_("TimeoutError".to_string()), ) .input(None) .output(None), ) .tags(vec![])]), LLMObsEventType::EVENTS, )); let mut configuration = datadog::Configuration::new(); configuration.set_unstable_operation_enabled("v2.CreateLLMObsExperimentEvents", true); let api = LLMObservabilityAPI::with_config(configuration); let resp = api .create_llm_obs_experiment_events("experiment_id".to_string(), body) .await; if let Ok(value) = resp { println!("{:#?}", value); } else { println!("{:#?}", resp.unwrap_err()); } } ``` #### Instructions First [install the library and its dependencies](https://docs.datadoghq.com/api/latest.md?code-lang=rust) and then save the example to `src/main.rs` and run following commands: DD_SITE="datadoghq.com" DD_API_KEY="" DD_APP_KEY="" cargo run ##### ```typescript /** * Push events for an LLM Observability experiment returns "Accepted" response */ import { client, v2 } from "@datadog/datadog-api-client"; const configuration = client.createConfiguration(); configuration.unstableOperations["v2.createLLMObsExperimentEvents"] = true; const apiInstance = new v2.LLMObservabilityApi(configuration); const params: v2.LLMObservabilityApiCreateLLMObsExperimentEventsRequest = { body: { data: { attributes: { metrics: [ { assessment: "pass", error: {}, label: "faithfulness", metricType: "score", spanId: "span-7a1b2c3d", tags: [], timestampMs: 1705314600000, }, ], spans: [ { datasetId: "9f64e5c7-dc5a-45c8-a17c-1b85f0bec97d", duration: 1500000000, meta: { error: { message: "Model response timed out", stack: `Traceback (most recent call last): File "main.py", line 10, in response = model.generate(input) File "model.py", line 45, in generate raise TimeoutError("Model response timed out") TimeoutError: Model response timed out`, type: "TimeoutError", }, input: undefined, output: undefined, }, name: "llm_call", projectId: "a33671aa-24fd-4dcd-9b33-a8ec7dde7751", spanId: "span-7a1b2c3d", startNs: 1705314600000000000, status: "ok", tags: [], traceId: "abc123def456", }, ], }, type: "events", }, }, experimentId: "experiment_id", }; apiInstance .createLLMObsExperimentEvents(params) .then((data: any) => { console.log( "API called successfully. Returned data: " + JSON.stringify(data) ); }) .catch((error: any) => console.error(error)); ``` #### Instructions First [install the library and its dependencies](https://docs.datadoghq.com/api/latest.md?code-lang=typescript) and then save the example to `example.ts` and run following commands: DD_SITE="datadoghq.com" DD_API_KEY="" DD_APP_KEY="" tsc "example.ts" {% /tab %}