Overview
Google Cloud Run is a fully managed serverless platform for deploying and scaling container-based applications. Datadog provides monitoring and log collection for Cloud Run through the Google Cloud integration. Datadog also provides a solution, now in public beta, for instrumenting your Cloud Run applications with a purpose-built Agent to enable tracing, custom metrics, and direct log collection.
This feature is in public beta. You can provide feedback through a
feedback form, or through your standard support channels. During the beta period, Cloud Run monitoring and APM tracing are available without a direct cost. Existing APM customers may incur increased span ingestion and volume costs.
Getting started
Prerequisites
Make sure you have a Datadog API Key and are using a programming language supported by a Datadog tracing library.
1. Install Agent
You can install the Agent using Dockerfile or a buildpack. If you use buildpack, you must install your tracing library first.
Install Agent with Dockerfile
You can instrument your application with a Datadog Agent by adding the following lines to your Dockerfile. You may need to adjust these examples depending on your existing Dockerfile setup.
# copy the Datadog `serverless-init` into your Docker image
COPY --from=datadog/serverless-init /datadog-init /app/datadog-init
# change the entrypoint to wrap your application into the Datadog serverless-init process
ENTRYPOINT ["/app/datadog-init"]
# optionally add Datadog tags
ENV DD_SERVICE=datadog-demo-run-go
ENV DD_ENV=datadog-demo
ENV DD_VERSION=1
# this env var is needed for trace propagation to work properly in Cloud Run.
# ensure to set this variable for all Datadog-instrumented downstream services.
ENV DD_TRACE_PROPAGATION_STYLE=datadog
# execute your binary application wrapped in the entrypoint. Adapt this line to your needs
CMD ["/path/to/your-go-binary"]
Instrument your application with the Datadog Agent by adding the following lines to your Dockerfile. You may need to adjust these examples depending on your existing Dockerfile setup.
# copy the Datadog `serverless-init` into your Docker image
COPY --from=datadog/serverless-init /datadog-init /app/datadog-init
# install the python tracing library here or in requirements.txt
RUN pip install --no-cache-dir ddtrace==1.7.3
# optionally add Datadog tags
ENV DD_SERVICE=datadog-demo-run-python
ENV DD_ENV=datadog-demo
ENV DD_VERSION=1
# this env var is needed for trace propagation to work properly in cloud run.
# ensure to set this variable for all Datadog-instrumented downstream services.
ENV DD_TRACE_PROPAGATION_STYLE=datadog
# change the entrypoint to wrap your application into the Datadog serverless-init process
ENTRYPOINT ["/app/datadog-init"]
# execute your binary application wrapped in the entrypoint, launched by the Datadog trace library. Adapt this line to your needs
CMD ["ddtrace-run", "python", "app.py"]
Instrument your application with the Datadog Agent by adding the following lines to your Dockerfile. You may need to adjust these examples depending on your existing Dockerfile setup.
# copy the Datadog `serverless-init` into your Docker image
COPY --from=datadog/serverless-init /datadog-init /app/datadog-init
# install the Datadog js tracing library, either here or in package.json
npm i dd-trace@2.2.0
# enable the Datadog tracing library
ENV NODE_OPTIONS="--require dd-trace/init"
# optionally add Datadog tags
ENV DD_SERVICE=datadog-demo-run-nodejs
ENV DD_ENV=datadog-demo
ENV DD_VERSION=1
# this env var is needed for trace propagation to work properly in cloud run.
# ensure to set this variable for all Datadog-instrumented downstream services.
ENV DD_TRACE_PROPAGATION_STYLE=datadog
# change the entrypoint to wrap your application into the Datadog serverless-init process
ENTRYPOINT ["/app/datadog-init"]
# execute your binary application wrapped in the entrypoint. Adapt this line to your needs
CMD ["/nodejs/bin/node", "/path/to/your/app.js"]
Instrument your application with the Datadog Agent by adding the following lines to your Dockerfile. You may need to adjust these examples depending on your existing Dockerfile setup.
# copy the Datadog `serverless-init` into your Docker image
COPY --from=datadog/serverless-init /datadog-init /app/datadog-init
# optionally add Datadog tags
ENV DD_SERVICE=datadog-demo-run-java
ENV DD_ENV=datadog-demo
ENV DD_VERSION=1
# this env var is needed for trace propagation to work properly in cloud run.
# ensure to set this variable for all Datadog-instrumented downstream services.
ENV DD_TRACE_PROPAGATION_STYLE=datadog
# change the entrypoint to wrap your application into the Datadog serverless-init process
ENTRYPOINT ["/app/datadog-init"]
# execute your binary application wrapped in the entrypoint. Adapt this line to your needs
CMD ["./mvnw", "spring-boot:run"]
Instrument your application with the Datadog Agent by adding the following lines to your Dockerfile. You may need to adjust these examples depending on your existing Dockerfile setup.
# copy the Datadog `serverless-init` into your Docker image
COPY --from=datadog/serverless-init /datadog-init /app/datadog-init
# optionally add Datadog tags
ENV DD_SERVICE=datadog-demo-run-dotnet
ENV DD_ENV=datadog-demo
ENV DD_VERSION=1
# this env var is needed for trace propagation to work properly in cloud run.
# ensure to set this variable for all Datadog-instrumented downstream services.
ENV DD_TRACE_PROPAGATION_STYLE=datadog
# change the entrypoint to wrap your application into the Datadog serverless-init process
ENTRYPOINT ["/app/datadog-init"]
# execute your binary application wrapped in the entrypoint. Adapt this line to your needs
CMD ["dotnet", "helloworld.dll"]
Instrument your application with the Datadog Agent by adding the following lines to your Dockerfile. You may need to adjust these examples depending on your existing Dockerfile setup.
# copy the Datadog `serverless-init` into your Docker image
COPY --from=datadog/serverless-init /datadog-init /app/datadog-init
# optionally add Datadog tags
ENV DD_SERVICE=datadog-demo-run-ruby
ENV DD_ENV=datadog-demo
ENV DD_VERSION=1
# this env var is needed for trace propagation to work properly in cloud run.
# ensure to set this variable for all Datadog-instrumented downstream services.
ENV DD_TRACE_PROPAGATION_STYLE=datadog
# change the entrypoint to wrap your application into the Datadog serverless-init process
ENTRYPOINT ["/app/datadog-init"]
# execute your binary application wrapped in the entrypoint. Adapt this line to your needs
CMD ["rails", "server", "-b", "0.0.0.0"] (adapt this line to your needs)
Install Agent with buildpack
Pack Buildpacks
provide a convenient way to package your container without using a Dockerfile. This example uses the Google Cloud container registry and Datadog serverless buildpack.
Note: Install the tracing library for your language before running the buildpack.
Build your application by running the following command:
pack build --builder=gcr.io/buildpacks/builder \
--buildpack from=builder \
--buildpack datadog/serverless-buildpack:latest \
gcr.io/YOUR_PROJECT/YOUR_APP_NAME
Note: Not compatible with Alpine.
2. Install tracing library
If you used buildpack, you can skip to configure your application.
Once the container is built and pushed to your registry, the last step is to set the required environment variables for the Datadog Agent:
DD_API_KEY
: Datadog API key, used to send data to your Datadog account. It should be configured as a Google Cloud Secret for privacy and safety issue.DD_SITE
: Datadog endpoint and website. Select your site on the right side of this page. Your site is:
.DD_TRACE_ENABLED
: set to true
to enable tracing
For more environment variables and their function, see Additional Configurations.
This command deploys the service and allows any external connection to reach it. Set DD_API_KEY
as an environment variable, and set your service listening to port 80.
gcloud run deploy APP_NAME --image=gcr.io/YOUR_PROJECT/APP_NAME \
--port=80 \
--update-env-vars=DD_API_KEY=$DD_API_KEY \
--update-env-vars=DD_TRACE_ENABLED=true \
--update-env-vars=DD_SITE='datadoghq.com' \
--allow-unauthenticated
3. Results
Once the deployment is completed, your metrics and traces are sent to Datadog. In Datadog, navigate to Infrastructure->Serverless to see your serverless metrics and traces.
Additional configurations
Advanced Tracing: The Datadog Agent already provides some basic tracing for popular frameworks. Follow the advanced tracing guide for more information.
Logs: If you use the Google Cloud integration, your logs are already being collected. Alternatively, you can set the DD_LOGS_ENABLED
environment variable to true
to capture application logs through the serverless instrumentation directly.
Custom Metrics: You can submit custom metrics using a DogStatsd client. For monitoring Cloud Run and other serverless applications, use distribution metrics. Distributions provide avg
, sum
, max
, min
, and count
aggregations by default. On the Metric Summary page, you can enable percentile aggregations (p50, p75, p90, p95, p99) and also manage tags. To monitor a distribution for a gauge metric type, use avg
for both the time and space aggregations. To monitor a distribution for a count metric type, use sum
for both the time and space aggregations.
Trace Propagation: In order to propagate trace context for distributed tracing, set the DD_TRACE_PROPAGATION_STYLE
environment variable to 'datadog'
for your Cloud Run app and any Datadog-instrumented services downstream of it.
Environment Variables
OpenTelemetry
Follow these steps to send OpenTelemetry (OTel) data to Datadog.
Tell OTel to export spans to Datadog serverless-init
.
// instrument.js
const { NodeTracerProvider } = require("@opentelemetry/sdk-trace-node");
const { OTLPTraceExporter } = require('@opentelemetry/exporter-trace-otlp-http');
const { Resource } = require('@opentelemetry/resources');
const { SemanticResourceAttributes } = require('@opentelemetry/semantic-conventions');
const { SimpleSpanProcessor } = require('@opentelemetry/sdk-trace-base');
const provider = new NodeTracerProvider({
resource: new Resource({
[ SemanticResourceAttributes.SERVICE_NAME ]: '<your-service-name>',
})
});
provider.addSpanProcessor(
new SimpleSpanProcessor(
new OTLPTraceExporter(
{ url: 'http://localhost:4318/v1/traces' },
),
),
);
provider.register();
Add OTel’s instrumentation for Express. This is akin to adding ddtrace
.
// instrument.js
const { ExpressInstrumentation } = require('@opentelemetry/instrumentation-express');
const { HttpInstrumentation } = require('@opentelemetry/instrumentation-http');
const { registerInstrumentations } = require('@opentelemetry/instrumentation');
registerInstrumentations({
instrumentations: [
new HttpInstrumentation(),
new ExpressInstrumentation(),
],
});
Add instrumentation at runtime. For instance, for Node.js, use NODE_OPTIONS
.
# Dockerfile
FROM node
WORKDIR /app
COPY package.json index.js instrument.js /app/
RUN npm i
ENV NODE_OPTIONS="--require ./instrument"
CMD npm run start
Add the Datadog serverless-init
.
# Dockerfile
COPY --from=datadog/serverless-init /datadog-init /app/datadog-init
ENTRYPOINT ["/app/datadog-init"]
Enable OTel in the Datadog serverless-init
using the DD_OTLP_CONFIG_RECEIVER_PROTOCOLS_HTTP_ENDPOINT
or DD_OTLP_CONFIG_RECEIVER_PROTOCOLS_GRPC_ENDPOINT
environment variable.
# Dockerfile
ENV DD_OTLP_CONFIG_RECEIVER_PROTOCOLS_HTTP_ENDPOINT="localhost:4318"
Troubleshooting
This integration depends on your runtime having a full SSL implementation. If you are using a slim image for Node, you may need to add the following command to your Dockerfile to include certificates.
RUN apt-get update && apt-get install -y ca-certificates
Further reading
Additional helpful documentation, links, and articles: