LLM Observability is not available in the US1-FED site.
Overview
LLM Observability categorizes spans by their span kind, which defines the type of work the span is performing. This can give you more granular insights on what operations are being performed by your LLM application.
LLM Observability supports the following span kinds:
| Kind | Represents | Root span? | Can have child spans? | Examples |
|---|
| LLM | A call to an LLM. | Yes | No | A call to a model, such as OpenAI GPT-4. |
| Workflow | Any predetermined sequence of operations which include LLM calls and any surrounding contextual operations. | Yes | Yes | A service that takes a URL and returns a summary of the page, requiring a tool call to fetch the page, some text processing tasks, and an LLM summarization. |
| Agent | A series of decisions and operations made by an autonomous agent, which usually consist of nested workflows, LLMs, tools, and task calls. | Yes | Yes | A chatbot that answers customer questions. |
| Tool | A call to external programs or services. | No | No | A call to a web search API or calculator. |
| Task | A standalone step that does not involve a call to an external service. | No | No | A data preprocessing step. |
| Embedding | A call to a model or function that returns an embedding. | No | Yes | A call to text-embedding-ada-002. |
| Retrieval | A data retrieval operation from an external knowledge base. | No | No | A call to a vector database that returns an array of ranked documents. |
For instructions on creating spans from your application, including code examples, see Tracing spans in the SDK documentation.
LLM span
LLM spans represent a call to an LLM where input and outputs are represented as text.
A trace can contain a single LLM span, in which case the trace represents an LLM inference operation.
LLM spans typically do not have child spans, as they are standalone operations representing a direct call to an LLM.
Workflow span
Workflow spans represent any static sequence of operations. Use workflows to group together an LLM call with its supporting contextual operations, such as tool calls, data retrievals, and other tasks.
Workflow spans are frequently the root span of a trace consisting of a standard sequence. For example, a function might take an arXiv paper link and return a summary. This process might involve a tool call to fetch the paper, some text-processing tasks, and an LLM summarization.
Workflow spans may have any spans as children, which represent child steps in the workflow sequence.
Agent span
Agent spans represent a dynamic sequence of operations where a large language model determines and executes operations based on the inputs. For example, an agent span might represent a series of reasoning steps controlled by a ReAct agent.
Agent spans are frequently the root span for traces representing autonomous agents or reasoning agents.
Agent spans may have any spans as children, which represent child steps orchestrated by a reasoning engine.
Tool spans represent a standalone step in a workflow or agent that involves a call to an external program or service, such as a web API or database.
Tool spans typically do not have child spans, as they are standalone operations representing a tool execution.
Task span
Task spans represent a standalone step in a workflow or agent that does not involve a call to an external service, such as a data sanitization step before a prompt is submitted to an LLM.
Task spans typically do not have child spans, as they are standalone steps in the workflow or agent.
Embedding span
Embedding spans are a subcategory of tool spans and represent a standalone call to an embedding model or function to create an embedding. For example, an embedding span could be used to trace a call to OpenAI’s embedding endpoint.
Embedding spans can have task spans as children, but typically do not have children.
Retrieval span
Retrieval spans are a subcategory of tool spans and represent a vector search operation involving a list of documents being returned from an external knowledge base. For example, a retrieval span could be used to trace a similarity search to a vector store to collect relevant documents for augmenting a user prompt for a given topic.
When used alongside embedding spans, retrieval spans can provide visibility into retrieval augmented generation (RAG) operations.
Retrieval spans typically do not have child spans, as they represent a standalone retrieval step.
