---
title: Fireworks AI for Agent Observability
description: Save your Fireworks AI API key for use in Agent Observability.
breadcrumbs: Docs > Integrations > Fireworks AI for Agent Observability
---

> For the complete documentation index, see [llms.txt](https://docs.datadoghq.com/llms.txt).

# Fireworks AI for Agent Observability
Integration version1.0.0
{% callout %}
# Important note for users on the following Datadog sites: us2.ddog-gov.com

{% alert level="info" %}
To find out if this integration is available in your organization, see your [Datadog Integrations](https://app.datadoghq.com/integrations) page or ask your organization administrator.

To initiate an exception request to enable this integration for your organization, email [support@ddog-gov.com](mailto:support@ddog-gov.com).
{% /alert %}

{% /callout %}

## Overview{% #overview %}

[Fireworks AI](https://fireworks.ai/) is an inference platform for open source and fine-tuned models. With the Fireworks AI integration, you can use Fireworks AI models in Datadog Agent Observability with your own Fireworks AI API key:

- **[Custom LLM-as-a-judge evaluators](https://docs.datadoghq.com/llm_observability/investigate/evaluations/llm_as_a_judge_evaluations.md)**: Use a Fireworks AI model as the judge that evaluates your application's spans, traces, or sessions.
- **[Playground](https://docs.datadoghq.com/llm_observability/improve/playground.md)**: Test prompts, tool calls, and structured outputs against Fireworks AI models.

You can select a model from the list that Fireworks AI returns for your API key, or enter a model name, such as the name of a deployed fine-tuned model.

Datadog sends these requests to Fireworks AI with your API key, so Fireworks AI bills the usage to your account.

## Setup{% #setup %}

### Installation{% #installation %}

No installation steps are required.

### Configuration{% #configuration %}

#### Create a Fireworks AI API key{% #create-a-fireworks-ai-api-key %}

1. In the Fireworks AI dashboard, open the [API keys](https://app.fireworks.ai/settings/users/api-keys) page.
1. Click **Create API key** and copy the key.

#### Add the Fireworks AI integration{% #add-the-fireworks-ai-integration %}

Adding an account requires the **Integrations Manage** permission.

1. In Datadog, open the [Fireworks AI for Agent Observability integration tile](https://app.datadoghq.com/integrations?integrationId=fireworks). You can also find the tile in [**Agent Observability** > **Settings** > **Integrations**](https://app.datadoghq.com/llm/settings/integrations).
1. On the **Configure** tab, click **Add new**.
1. Enter an account name and the API key from the previous step.
1. Click the check mark to save the account.

**Note**: Datadog does not validate the API key when you save the account. To replace a key, edit the account and save the new key.

### Validation{% #validation %}

1. In the [Playground](https://app.datadoghq.com/llm/playground), open **Model configuration** from the toolbar.
1. Under **Provider**, select **Fireworks AI**. Then select your account under **Account** and a model under **Model**, and click **Done**.
1. Enter a prompt and click **Run**.

A response from the model confirms that the API key works.

## Use Fireworks AI models{% #use-fireworks-ai-models %}

### Custom LLM-as-a-judge evaluators{% #custom-llm-as-a-judge-evaluators %}

When you create a custom LLM-as-a-judge evaluator from the [Evaluators page](https://app.datadoghq.com/llm/evaluations):

1. Next to **Model**, click the model button to open **Model configuration**.
1. Under **Account**, select **Fireworks AI** > your account. If no Fireworks AI account is listed, select **+ Add new account** > **+ Fireworks AI** to add one.
1. Under **Model**, select a model, or enter a model name such as `accounts/<ACCOUNT_ID>/models/<MODEL_ID>` and select **Use model ID**. For a model on a dedicated deployment, use the name format from the [Fireworks AI deployments guide](https://docs.fireworks.ai/guides/ondemand-deployments).
1. Click **Confirm**.

Each evaluator stores its own provider, account, and model.

### Playground{% #playground %}

After you select a Fireworks AI model in the Playground, click **Get code** to export the prompt as code. The code reads the API key from the `FIREWORKS_API_KEY` environment variable.

### Limitations{% #limitations %}

- Among evaluators, only custom LLM-as-a-judge evaluators can use Fireworks AI. [Managed evaluators](https://docs.datadoghq.com/llm_observability/investigate/evaluations/managed_evaluations.md) do not.
- Fireworks AI judges evaluate text only. Datadog does not send images or audio from spans to Fireworks AI. For providers that support them, see [Image and audio evaluators](https://docs.datadoghq.com/llm_observability/investigate/evaluations/llm_as_a_judge_evaluations/image_and_audio_evaluations.md).
- The model list might not include every model, such as a model on a dedicated deployment. If a model is missing, enter its name.
- Fireworks AI is not available for HIPAA-enabled organizations.

## Data Collected{% #data-collected %}

### Metrics{% #metrics %}

This integration does not collect metrics.

### Service Checks{% #service-checks %}

This integration does not include service checks.

### Events{% #events %}

This integration does not collect events.

## Support{% #support %}

Need help? Contact [Datadog support](https://docs.datadoghq.com/help/).
