Trace an LLM Application in AWS Lambda

This product is not supported for your selected Datadog site. ().

To instrument an existing AWS Lambda function with Agent Observability, you can use the Datadog Extension and respective language layers.

  1. Open a Cloudshell in the AWS console.
  2. Install the Datadog CLI client
npm install -g @datadog/datadog-ci
  1. Set the Datadog API key and site
export DD_API_KEY=<YOUR_DATADOG_API_KEY>
export DD_SITE=<YOUR_DATADOG_SITE>

If you already have or prefer to use a secret in Secrets Manager, you can set the API key by using the secret ARN:

export DATADOG_API_KEY_SECRET_ARN=<DATADOG_API_KEY_SECRET_ARN>
  1. Instrument your Lambda function with Agent Observability (this requires at least version 77 of the Datadog Extension layer)

    datadog-ci lambda instrument -f <YOUR_LAMBDA_FUNCTION_NAME> -r <AWS_REGION> -v 127 -e 99 --llmobs <YOUR_LLMOBS_ML_APP>
    
    datadog-ci lambda instrument -f <YOUR_LAMBDA_FUNCTION_NAME> -r <AWS_REGION> -v 142 -e 99 --llmobs <YOUR_LLMOBS_ML_APP>
    
    datadog-ci lambda instrument -f <YOUR_LAMBDA_FUNCTION_NAME> -r <AWS_REGION> -v 27 -e 99 --llmobs <YOUR_LLMOBS_ML_APP>
    

  2. Invoke your Lambda function and verify that Agent Observability traces are visible in the Datadog UI.

Manually flush Agent Observability traces by using the flush method before the Lambda function returns.

from ddtrace.llmobs import LLMObs
def handler():
  # function body
  LLMObs.flush()
import tracer from 'dd-trace';
const llmobs = tracer.llmobs;

export const handler = async (event) => {
  // your function body
  llmobs.flush();
};

Further Reading

Additional helpful documentation, links, and articles: