Setup

1. Install the Datadog Node.js tracer.

   1. In your main application, add dd-trace-js.

      Copy

      npm install dd-trace --save

   2. Add the following to your application code to initialize the tracer:

      Copy

      const tracer = require('dd-trace').init({
       logInjection: true,
      });

   For more information, see Tracing Node.js applications.

2. Install serverless-init as a sidecar.

   Setup

   Install the Datadog CLI

   npm install -g @datadog/datadog-ci @datadog/datadog-ci-plugin-cloud-run
   

   Install the gcloud CLI and authenticate with gcloud auth login.

   Configuration

   Configure the Datadog site and Datadog API key, and define the service name to use in Datadog.

   export DATADOG_SITE="<DATADOG_SITE>"
   export DD_API_KEY="<DD_API_KEY>"
   export DD_SERVICE="<SERVICE_NAME>"
   

   Instrument

   If you are new to Datadog serverless monitoring, launch the Datadog CLI in interactive mode to guide your first installation for a quick start.

   datadog-ci cloud-run instrument -i
   

   To set up the Datadog sidecar for your applications, run the instrument command after your normal deployment. You can specify multiple services to instrument by passing multiple --service flags.

   datadog-ci cloud-run instrument --project <GCP-PROJECT-ID> --service <CLOUD-RUN-SERVICE-NAME> --region <GCP-REGION>
   

   You can pin to a specific image with the --sidecar-image flag. See the latest releases on Docker Hub.

   Additional parameters can be found in the CLI documentation.

   The Datadog Terraform module for Google Cloud Run wraps the google_cloud_run_v2_service resource and automatically configures your Cloud Run app for Datadog Serverless Monitoring by adding required environment variables and the serverless-init sidecar.

   If you don’t already have Terraform set up, install Terraform, create a new directory, and make a file called main.tf.

   Then, add the following to your Terraform configuration, updating it as necessary based on your needs:

   variable "datadog_api_key" {
     description = "Your Datadog API key"
     type        = string
     sensitive   = true
   }
   
   module "my-cloud-run-app" {
     source  = "DataDog/cloud-run-datadog/google"
     version = "~> 1.0"
   
     project  = "my-gcp-project"
     name     = "my-cloud-run-app"
     location = "us-central1"
   
     datadog_api_key = var.datadog_api_key
     datadog_service = "test-service" // your application service
     datadog_version = "0.0.0" // your code version
     datadog_env     = "prod" // your application environment
     
     datadog_enable_logging = true
   
     deletion_protection = false
     build_config = {
       function_target          = "helloHttp" // your function entry point
       image_uri                = "us-docker.pkg.dev/cloudrun/container/hello"
       base_image               = "us-central1-docker.pkg.dev/serverless-runtimes/google-22-full/runtimes/your-runtime" // base image for your runtime
       enable_automatic_updates = true
     }
     template = {
       containers = [
         {
           name  = "main"
           image = "us-docker.pkg.dev/cloudrun/container/hello"
           base_image_uri = "us-central1-docker.pkg.dev/serverless-runtimes/google-22-full/runtimes/your-runtime" // base image for your runtime
           resources = {
             limits = {
               cpu    = "1"
               memory = "512Mi"
             }
           }
           ports = {
             container_port = 8080
           }
           env = [
             { name = "DD_TRACE_ENABLED", value = "true" },
           ]
         },
       ]
     }
   }
   

   See the Environment Variables for more information on the configuration options available through the env.

   Ensure the container port for the main container is the same as the one exposed in your Dockerfile/service.

   If you haven’t already, initialize your Terraform project:

   terraform init
   

   To deploy your app, run:

   terraform apply
   

   After deploying your Cloud Run app, you can manually modify your app’s settings to enable Datadog monitoring.

   1. Create a Volume with In-Memory volume type.

   2. Add a new container with image URL: gcr.io/datadoghq/serverless-init:<YOUR_TAG>. See the latest releases on Docker Hub to pin a specific version.

   3. Add the volume mount to every container in your application. Choose a path such as /shared-volume, and remember it for the next step.

   4. Add the following environment variables to your serverless-init sidecar container:

      • DD_SERVICE: A name for your service. For example, gcr-sidecar-test.
      • DD_ENV: A name for your environment. For example, dev.
      • DD_SERVERLESS_LOG_PATH: Your log path. For example, /shared-volume/logs/*.log. The path must begin with the mount path you defined in the previous step.
      • DD_API_KEY: Your Datadog API key.
      • FUNCTION_TARGET: The entry point of your function. For example, Main.

      For a list of all environment variables, including additional tags, see Environment variables.

3. Set up logs.

   In the previous step, you created a shared volume. You may have also set the DD_SERVERLESS_LOG_PATH environment variable, which defaults to /shared-volume/logs/app.log.

   In this step, configure your logging library to write logs to the file set in DD_SERVERLESS_LOG_PATH. In Node.js, Datadog recommend writing logs in a JSON format. For example, you can use a third-party logging library such as winston:

   Copy

   const tracer = require('dd-trace').init({
     logInjection: true,
   });
   const { createLogger, format, transports } = require('winston');
   
   const LOG_FILE = "/shared-volume/logs/app.log"
   
   const logger = createLogger({
     level: 'info',
     exitOnError: false,
     format: format.json(),
     transports: [
       new transports.File({ filename: LOG_FILE }),
       new transports.Console()
     ],
   });
   
   logger.info('Hello world!');

   Datadog recommends setting the environment variables DD_LOGS_INJECTION=true (in your main container) and DD_SOURCE=nodejs (in your sidecar container) to enable advanced Datadog log parsing.

   For more information, see Correlating Node.js Logs and Traces.

4. Add a service label in Google Cloud. In your Cloud Run service’s info panel, add a label with the following key and value:

   Key	Value
   service	The name of your service. Matches the value provided as the DD_SERVICE environment variable.

   See Configure labels for services in the Cloud Run documentation for instructions.

5. Send custom metrics.

   To send custom metrics, view code examples. In Serverless Monitoring, only the distribution metric type is supported.

Environment variables

Distributed tracing with Pub/Sub

To get end-to-end distributed traces between Pub/Sub producers and Cloud Run functions, configure your push subscriptions with the --push-no-wrapper and --push-no-wrapper-write-metadata flags. This moves message attributes from the JSON body to HTTP headers, allowing Datadog to extract producer trace context and create proper span links.

For more information, see Producer-aware tracing for Google Cloud Pub/Sub and Cloud Run and Payload unwrapping in the Google Cloud documentation.

Configure push subscriptions for full trace visibility

Create a new push subscription:

gcloud pubsub subscriptions create order-processor-sub \
  --topic=orders \
  --push-endpoint=https://order-processor-xyz.run.app/pubsub \
  --push-no-wrapper \
  --push-no-wrapper-write-metadata

Update an existing push subscription:

gcloud pubsub subscriptions update order-processor-sub \
  --push-no-wrapper \
  --push-no-wrapper-write-metadata

Configure Eventarc Pub/Sub triggers

Eventarc Pub/Sub triggers use push subscriptions as the underlying delivery mechanism. When you create an Eventarc trigger, GCP automatically creates a managed push subscription. However, Eventarc does not expose --push-no-wrapper-write-metadata as a trigger creation parameter, so you must manually update the auto-created subscription.

1. Create the Eventarc trigger:

   Copy

   gcloud eventarc triggers create order-processor-trigger \
     --destination-run-service=order-processor \
     --destination-run-region=us-central1 \
     --event-filters="type=google.cloud.pubsub.topic.v1.messagePublished" \
     --event-filters="topic=projects/my-project/topics/orders" \
     --location=us-central1

2. Find the auto-created subscription:

   Copy

   gcloud pubsub subscriptions list \
     --filter="topic:projects/my-project/topics/orders" \
     --format="table(name,pushConfig.pushEndpoint)"

   Example output:

   NAME                                                          PUSH_ENDPOINT
   eventarc-us-central1-order-processor-trigger-abc-sub-def      https://order-processor-xyz.run.app
   

3. Update the subscription for trace propagation:

   Copy

   gcloud pubsub subscriptions update \
     eventarc-us-central1-order-processor-trigger-abc-sub-def \
     --push-no-wrapper \
     --push-no-wrapper-write-metadata

Troubleshooting

This integration depends on your runtime having a full SSL implementation. If you are using a slim image, you may need to add the following command to your Dockerfile to include certificates:

RUN apt-get update && apt-get install -y ca-certificates

To have your Cloud Run services appear in the software catalog, you must set the DD_SERVICE, DD_VERSION, and DD_ENV environment variables.

Further reading

Additional helpful documentation, links, and articles:

Tracing Node.js ApplicationsDOCUMENTATION Correlating Node.js Logs and TracesDOCUMENTATION