--- title: Prompt Management description: >- Create, version, and retrieve managed prompts in Python applications with Prompt Management. breadcrumbs: Docs > Agent Observability > Monitoring > Prompt Management --- > For the complete documentation index, see [llms.txt](https://docs.datadoghq.com/llms.txt). # Prompt Management {% callout %} # Important note for users on the following Datadog sites: app.ddog-gov.com, us2.ddog-gov.com {% alert level="danger" %} This product is not supported for your selected [Datadog site](https://docs.datadoghq.com/getting_started/site.md). ({% placeholder "user-datadog-site-name" /%}). {% /alert %} {% /callout %} {% callout %} # Important note for users on the following Datadog sites: app.datadoghq.com, us3.datadoghq.com, us5.datadoghq.com, app.datadoghq.eu, ap1.datadoghq.com, ap2.datadoghq.com, uk1.datadoghq.com {% callout %} ##### Join the Preview! Prompt Management is in Preview. {% /callout %} {% /callout %} ## Overview{% #overview %} Prompt Management provides a centralized registry for the prompts used by your LLM applications. Instead of hardcoding prompt templates in application code or configuration files, create, version, and update prompts through Agent Observability, then retrieve them at runtime. Runtime retrieval is supported in Python through the `ddtrace` SDK. Prompt retrieval and Prompt Tracking are separate: `LLMObs.get_prompt()` can retrieve a managed prompt without enabling Agent Observability, but Agent Observability must be enabled to create LLM spans and associate prompt metadata with them. Prompt Management works alongside [Prompt Tracking](https://docs.datadoghq.com/llm_observability/monitoring/prompt_tracking.md). When Agent Observability is enabled, managed prompts passed directly to supported, automatically instrumented LLM calls are associated with the resulting spans. ## Prerequisites{% #prerequisites %} - Python 3.9 or later. - Your [Datadog site](https://docs.datadoghq.com/getting_started/site.md) and a [Datadog API key](https://docs.datadoghq.com/account_management/api-app-keys.md#api-keys). The API key is required for prompt retrieval even if traces are sent through the Datadog Agent. - A [Datadog application key](https://docs.datadoghq.com/account_management/api-app-keys.md#application-keys) with the `llm_observability_read`, `feature_flag_config_read`, and `feature_flag_environment_config_read` permissions to resolve prompts by environment. If you select an existing application key in Datadog, ensure that it has these permissions. - To manage prompts through the API or Python SDK, the application key also requires the `llm_observability_write` and `feature_flag_config_write` permissions. ## Install the SDK{% #install-the-sdk %} Install or upgrade the latest `ddtrace` package in the Python environment used by your application: ```shell pip install --upgrade ddtrace ``` ## Use a managed prompt in Python{% #use-a-managed-prompt-in-python %} ### Integrate Prompt Management with a coding agent{% #integrate-prompt-management-with-a-coding-agent %} Integrate a managed prompt with a coding agent of your choice by pasting in the following prompt: ```text Follow the instructions at https://docs.datadoghq.com/llm_observability/instrumentation/agentic.md to integrate the Datadog managed prompt into this application for environment and track its use in Agent Observability. Prompt variables: When configuring the environment, use the following values: DD_SITE= DD_ENV= ``` Optionally, append selected Datadog credentials so the coding agent can configure and verify the integration in the same session: ```text Selected Datadog credentials: DD_API_KEY= DD_APP_KEY= Treat these values as secrets and handle them according to the linked guide. Do not repeat or expose them. ``` **Note:** Including the API and application keys in the prompt is optional and is not required for the coding agent to integrate Prompt Management. Include them only in a trusted coding-agent session. After the integration is complete, run your application and trigger the modified LLM flow. Return to the prompt page to view usage; new prompt calls may take a minute to appear. ### Configure prompt retrieval{% #configure-prompt-retrieval %} Provide the Datadog site, credentials, and deployment environment through the configuration and secret-management workflow already used by your application. For example, use the application's environment file, Docker Compose or Kubernetes configuration, deployment platform, or secret manager. At runtime, the following environment variables must be set before importing `ddtrace`: ```shell export DD_SITE="" export DD_API_KEY="" export DD_APP_KEY="" export DD_ENV="" ``` `DD_ENV` selects the environment used to resolve the prompt version and must match an environment where the prompt is deployed. ### Retrieve, format, and use a prompt{% #retrieve-format-and-use-a-prompt %} Preserve the prompt already used by your application as the fallback. The fallback keeps the application working if registry, environment-resolution, network, or server failures occur. The following example retrieves and formats a chat prompt, then passes the formatted messages directly to OpenAI: ```python from ddtrace.llmobs import LLMObs from openai import OpenAI default_messages = [ {"role": "system", "content": "You are a support agent for {{company}}."}, {"role": "user", "content": "{{question}}"}, ] variables = { "company": "Acme Inc.", "question": "How do I reset my password?", } prompt = LLMObs.get_prompt( "customer-support-greeting", fallback=default_messages, ) messages = prompt.format(**variables) client = OpenAI() response = client.chat.completions.create( model="gpt-4o", messages=messages, ) ``` `prompt.format()` returns a string for a text prompt and a list of messages for a chat prompt. Pass the formatted value to the corresponding text or messages parameter of your LLM provider call. If retrieval fails and no fallback is provided, `get_prompt()` raises a `ValueError`. A fallback does not replace authentication: `DD_API_KEY` is always required, and `DD_APP_KEY` is also required when `DD_ENV` is set. Managed prompts cannot reference other managed prompts in their templates. To compose prompts, combine them in application code or manage the final provider-facing prompt as a single prompt. ### Select a version{% #select-a-version %} Without `DD_ENV`, `get_prompt()` retrieves the latest prompt version: ```python prompt = LLMObs.get_prompt("customer-support-greeting") ``` With `DD_ENV`, `get_prompt()` resolves the prompt version for that environment. This requires `DD_APP_KEY` with the read permissions listed in Prerequisites. To retrieve an exact numeric version independently of `DD_ENV`, pass `version`: ```python prompt = LLMObs.get_prompt("customer-support-greeting", version=2) ``` The `version` argument takes precedence over environment resolution. ### Track prompt usage{% #track-prompt-usage %} To associate a managed prompt with an LLM span, [enable Agent Observability](https://docs.datadoghq.com/llm_observability/instrumentation/sdk.md?tab=python) and run the application with automatic instrumentation through its existing execution workflow. If the application receives its configuration before the Python process starts, use `ddtrace-run`. For example, the equivalent shell command is: ```shell DD_SITE="" \ DD_API_KEY="" \ DD_APP_KEY="" \ DD_ENV="" \ DD_SERVICE="" \ DD_LLMOBS_ENABLED=1 \ ddtrace-run python app.py ``` If the application loads its configuration in Python, load the configuration first, then import `ddtrace.auto` before importing the LLM provider or other application modules: ```python from dotenv import load_dotenv load_dotenv() import ddtrace.auto from ddtrace.llmobs import LLMObs from openai import OpenAI ``` Run this setup with the application's normal Python command, such as `python app.py`. Do not also use `ddtrace-run`; it initializes `ddtrace` before the application can load its configuration. If the application does not send data through a Datadog Agent, also set `DD_LLMOBS_AGENTLESS_ENABLED=1`. For a [supported automatically instrumented provider](https://docs.datadoghq.com/llm_observability/instrumentation/auto_instrumentation.md?tab=python), pass the value returned by `prompt.format()` directly to the provider call, as shown in Retrieve, format, and use a prompt. This automatically associates the managed prompt with the resulting span. Copying, rebuilding, or converting the formatted value can discard its prompt-tracking metadata. For example, concatenating a managed system prompt with a user question creates a new string without that metadata. Use `LLMObs.annotation_context()` to associate the managed prompt with the resulting LLM span: ```python prompt = LLMObs.get_prompt( "customer-support-system-prompt", fallback="You are a helpful support agent writing for a {{audience}} audience.", ) variables = {"audience": audience} system_prompt = prompt.format(**variables) combined_prompt = f"{system_prompt}\n\nUser question: {question}" with LLMObs.annotation_context( prompt=prompt.to_annotation_dict(**variables), ): response = client.responses.create( model="gpt-4o", input=combined_prompt, ) ``` Pass the same variables to `to_annotation_dict()` that you pass to `format()` so that the tracked prompt includes the values used for that call. `annotation_context()` associates metadata with an LLM span created inside the context; it does not create the span. For providers that are not automatically instrumented, first [manually instrument the LLM call](https://docs.datadoghq.com/llm_observability/instrumentation/sdk.md?tab=python#manual-instrumentation) to create an LLM span. An explicit `annotation_context()` takes precedence over automatic prompt tracking. See [Prompt Tracking](https://docs.datadoghq.com/llm_observability/monitoring/prompt_tracking.md) for more information. ## Create and manage prompts{% #create-and-manage-prompts %} Create prompts and publish new versions in the Prompts UI, through the Python SDK, or through the API. ### Create a prompt{% #create-a-prompt %} #### Promote a tracked prompt{% #promote-a-tracked-prompt %} To promote a prompt already tracked in Agent Observability to a managed prompt, navigate to the Prompts page, open the prompt, and click Register. You can then update the prompt in the UI and retrieve it at runtime. #### In the UI from scratch{% #in-the-ui-from-scratch %} Navigate to the Prompts page and click + New Prompt. In the Prompt Editor: 1. Add one or more messages and assign each a role: System, User, or Assistant. 1. Use `{{variable_name}}` syntax in any message to add dynamic content. 1. Optional: Click Run to test the prompt with sample values. 1. Click Save Prompt to open the save dialog. Structure the prompt so the user query and context are injected as variables: {% image source="https://docs.dd-static.net/images/llm_observability/monitoring/prompt-creation.f3c865986e7921f226b91d67a937f9c6.png?auto=format&fit=max&w=850 1x, https://docs.dd-static.net/images/llm_observability/monitoring/prompt-creation.f3c865986e7921f226b91d67a937f9c6.png?auto=format&fit=max&w=850&dpr=2 2x" alt="The Playground with a System Prompt message reading 'You are a support agent for {{company}}' and a User Prompt message containing {{question}}, with the Save Prompt button in the top right." /%} In the save dialog: | Field | Description | | ----------- | --------------------------------------------------------------------------------------------------------------------------------------- | | Prompt ID | A unique identifier for the prompt, such as `customer-support-greeting`. Use this ID to retrieve the prompt with `LLMObs.get_prompt()`. | | Description | Optional notes about this version. | | Deployment | The environment to which this version is deployed. | Click Create Prompt to save the prompt to the registry. ### Update, list, and delete prompts{% #update-list-and-delete-prompts %} #### In the UI{% #in-the-ui %} Open a prompt in the Prompts page to: - **Create a new version**: Click Edit and update the messages in the Prompt Editor. - **Deploy a version to another environment**: Select a version and update its Deployment environments. - **Delete a prompt**: Select Delete from the prompt's options menu. This removes the prompt and its version history from the registry. ### Use the Python SDK{% #use-the-python-sdk %} Use `LLMObs.create_prompt()` to create a prompt and deploy its first version to one or more environments. The `env_ids` values are Feature Flags environment IDs, which you can obtain from the [List environments API](https://docs.datadoghq.com/api/latest/feature-flags/list-environments.md): ```python from ddtrace.llmobs import LLMObs chat_template = [ {"role": "system", "content": "You are a support agent for {{company}}."}, {"role": "user", "content": "{{question}}"}, ] created_prompt = LLMObs.create_prompt( "customer-support-greeting", chat_template, env_ids=[""], ) ``` To publish and deploy another version, use `LLMObs.create_prompt_version()`: ```python created_version = LLMObs.create_prompt_version( "customer-support-greeting", updated_chat_template, env_ids=[""], ) ``` Treat prompt creation, versioning, and deployment as setup operations. Do not perform them during application startup or from a request path. At runtime, retrieve deployed prompts with `LLMObs.get_prompt()`. These methods require the API and application key permissions listed in Prerequisites. Use `LLMObs.list_prompts()` and `LLMObs.list_prompt_versions()` to inspect managed prompts, `LLMObs.update_prompt()` and `LLMObs.update_prompt_version()` to update metadata or deployments, and `LLMObs.delete_prompt()` to delete a prompt and all of its versions. ### Use the API{% #use-the-api %} Use the Prompt Management API to create, retrieve, update, and delete prompts and prompt versions. See the [LLM Observability API reference](https://docs.datadoghq.com/api/latest/llm-observability.md) for endpoint schemas, request media types, and examples. ## Further reading{% #further-reading %} - [Prompt Tracking](https://docs.datadoghq.com/llm_observability/monitoring/prompt_tracking.md) - [Playground](https://docs.datadoghq.com/llm_observability/playground.md) - [Agent Observability SDK](https://docs.datadoghq.com/llm_observability/instrumentation/sdk.md?tab=python)