Span Kinds

LLM Observability is not available in the US1-FED site.

LLM Observability is in public beta.

Overview

LLM Observability categorizes spans by their span kind, which defines the type of work the span is performing. This can give you more granular insights on what operations are being performed by your LLM application.

LLM Observability supports the following span kinds:

KindRepresentsRoot span?Can have child spans?Examples
LLMA call to an LLM.YesNoA call to a model, such as OpenAI GPT-4.
WorkflowAny predetermined sequence of operations which include LLM calls and any surrounding contextual operations.YesYesA service that takes a URL and returns a summary of the page, requiring a tool call to fetch the page, some text processing tasks, and an LLM summarization.
AgentA series of decisions and operations made by an autonomous agent, which usually consist of nested workflows, LLMs, tools, and task calls.YesYesA chatbot that answers customer questions.
ToolA call to external programs or services.NoNoA call to a web search API or calculator.
TaskA standalone step that does not involve a call to an external service.NoNoA data preprocessing step.
EmbeddingA call to a model or function that returns an embedding.NoYesA call to text-embedding-ada-002.
RetrievalA data retrieval operation from an external knowledge base.NoNoA call to a vector database that returns an array of ranked documents.

For instructions on creating spans from your application, including code examples, see Tracing spans in the SDK documentation.

LLM span

LLM spans represent a call to an LLM where input and outputs are represented as text.

A trace can contain a single LLM span, in which case the trace represents an LLM inference operation.

LLM spans typically do not have child spans, as they are standalone operations representing a direct call to an LLM.

Workflow span

Workflow spans represent any static sequence of operations. Use workflows to group together an LLM call with its supporting contextual operations, such as tool calls, data retrievals, and other tasks.

Workflow spans are frequently the root span of a trace consisting of a standard sequence. For example, a function might take an arXiv paper link and return a summary. This process might involve a tool call to fetch the paper, some text-processing tasks, and an LLM summarization.

Workflow spans may have any spans as children, which represent child steps in the workflow sequence.

Agent span

Agent spans represent a dynamic sequence of operations where a large language model determines and executes operations based on the inputs. For example, an agent span might represent a series of reasoning steps controlled by a ReAct agent.

Agent spans are frequently the root span for traces representing autonomous agents or reasoning agents.

Agent spans may have any spans as children, which represent child steps orchestrated by a reasoning engine.

Tool span

Tool spans represent a standalone step in a workflow or agent that involves a call to an external program or service, such as a web API or database.

Tool spans typically do not have child spans, as they are standalone operations representing a tool execution.

Task span

Task spans represent a standalone step in a workflow or agent that does not involve a call to an external service, such as a data sanitization step before a prompt is submitted to an LLM.

Task spans typically do not have child spans, as they are standalone steps in the workflow or agent.

Embedding span

Embedding spans are a subcategory of tool spans and represent a standalone call to an embedding model or function to create an embedding. For example, an embedding span could be used to trace a call to OpenAI’s embedding endpoint.

Embedding spans can have task spans as children, but typically do not have children.

Retrieval span

Retrieval spans are a subcategory of tool spans and represent a vector search operation involving a list of documents being returned from an external knowledge base. For example, a retrieval span could be used to trace a similarity search to a vector store to collect relevant documents for augmenting a user prompt for a given topic.

When used alongside embedding spans, retrieval spans can provide visibility into retrieval augmented generation (RAG) operations.

Retrieval spans typically do not have child spans, as they represent a standalone retrieval step.