Setup

1. Install the Datadog Node.js tracer.

   1. In your main application, add dd-trace-js.

      Copy

      npm install dd-trace --save

   2. Add the following to your application code to initialize the tracer:

      Copy

      const tracer = require('dd-trace').init({
       logInjection: true,
      });

   3. Set the following environment variable to specify that the dd-trace/init module is required when the Node.js process starts:

      Copy

      ENV NODE_OPTIONS="--require dd-trace/init"

   For more information, see Tracing Node.js applications.

2. Install serverless-init as a sidecar.

   Datadog publishes new releases of the serverless-init container image to Google’s gcr.io, AWS’s ECR, and on Docker Hub:

   hub.docker.com	gcr.io	public.ecr.aws
   datadog/serverless-init	gcr.io/datadoghq/serverless-init	public.ecr.aws/datadog/serverless-init

   Images are tagged based on semantic versioning, with each new version receiving three relevant tags:

   • 1, 1-alpine: use these to track the latest minor releases, without breaking changes
   • 1.x.x, 1.x.x-alpine: use these to pin to a precise version of the library
   • latest, latest-alpine: use these to follow the latest version release, which may include breaking changes
   Setup

   Install the Datadog CLI

   npm install -g @datadog/datadog-ci @datadog/datadog-ci-plugin-cloud-run
   

   Install the gcloud CLI and authenticate with gcloud auth login.

   Configuration

   Configure the Datadog site and Datadog API key, and define the service name to use in Datadog.

   export DATADOG_SITE="<DATADOG_SITE>"
   export DD_API_KEY="<DD_API_KEY>"
   export DD_SERVICE="<SERVICE_NAME>"
   

   Instrument

   If you are new to Datadog serverless monitoring, launch the Datadog CLI in interactive mode to guide your first installation for a quick start.

   datadog-ci cloud-run instrument -i
   

   To set up the Datadog sidecar for your applications, run the instrument command after your normal deployment. You can specify multiple services to instrument by passing multiple --service flags.

   datadog-ci cloud-run instrument --project <GCP-PROJECT-ID> --service <CLOUD-RUN-SERVICE-NAME> --region <GCP-REGION>
   

   You can pin to a specific image with the --sidecar-image flag. See the latest releases on Docker Hub.

   Additional parameters can be found in the CLI documentation.

   The Datadog Terraform module for Google Cloud Run wraps the google_cloud_run_v2_service resource and automatically configures your Cloud Run app for Datadog Serverless Monitoring by adding required environment variables and the serverless-init sidecar.

   If you don’t already have Terraform set up, install Terraform, create a new directory, and make a file called main.tf.

   Then, add the following to your Terraform configuration, updating it as necessary based on your needs:

   variable "datadog_api_key" {
     description = "Your Datadog API key"
     type        = string
     sensitive   = true
   }
   
   module "my-cloud-run-app" {
     source  = "DataDog/cloud-run-datadog/google"
     version = "~> 1.0"
   
     project  = "my-gcp-project"
     name     = "my-cloud-run-app"
     location = "us-central1"
   
     datadog_api_key = var.datadog_api_key
     datadog_service = "test-service" // your application service
     datadog_version = "0.0.0" // your code version
     datadog_env     = "prod" // your application environment
     
     datadog_enable_logging = true
   
     deletion_protection = false
     
     template = {
       containers = [
         {
           name  = "main"
           image = "us-docker.pkg.dev/cloudrun/container/hello"
           
           resources = {
             limits = {
               cpu    = "1"
               memory = "512Mi"
             }
           }
           ports = {
             container_port = 8080
           }
           env = [
             { name = "DD_TRACE_ENABLED", value = "true" },
           ]
         },
       ]
     }
   }
   

   See the Environment Variables for more information on the configuration options available through the env.

   Ensure the container port for the main container is the same as the one exposed in your Dockerfile/service.

   If you haven’t already, initialize your Terraform project:

   terraform init
   

   To deploy your app, run:

   terraform apply
   

   Create a YAML file that contains your configuration. You can use the following example and adapt it to your needs:

   apiVersion: serving.knative.dev/v1
   kind: Service
   metadata:
     name: '<SERVICE_NAME>'
     labels:
       cloud.googleapis.com/location: '<LOCATION>'
       service: '<SERVICE_NAME>'
   spec:
     template:
       metadata:
         labels:
           service: '<SERVICE_NAME>'
         annotations:
           # The maximum number of instances that can be created for this service.
           # https://cloud.google.com/run/docs/reference/rest/v1/RevisionTemplate
           autoscaling.knative.dev/maxScale: '100'
           # The startup CPU boost feature for revisions provides additional CPU during
           # instance startup time and for 10 seconds after the instance has started.
           # https://cloud.google.com/run/docs/configuring/services/cpu#startup-boost
           run.googleapis.com/startup-cpu-boost: 'true'
       spec:
         containers:
           - env:
               - name: DD_SERVICE
                 value: '<SERVICE_NAME>'
             image: '<CONTAINER_IMAGE>'
             name: run-sidecar-1
             ports:
               - containerPort: 8080
                 name: http1
             resources:
               limits:
                 cpu: 1000m
                 memory: 512Mi
             startupProbe:
               failureThreshold: 1
               periodSeconds: 240
               tcpSocket:
                 port: 8080
               timeoutSeconds: 240
             volumeMounts:
               - mountPath: /shared-volume
                 name: shared-volume
           - env:
               - name: DD_SERVERLESS_LOG_PATH
                 value: shared-volume/logs/*.log
               - name: DD_SITE
                 value: '<DATADOG_SITE>'
               - name: DD_ENV
                 value: '<ENV>'
               - name: DD_API_KEY
                 value: '<API_KEY>'
               - name: DD_SERVICE
                 value: '<SERVICE_NAME>'
               - name: DD_VERSION
                 value: '<VERSION>'
               - name: DD_LOG_LEVEL
                 value: debug
               - name: DD_LOGS_INJECTION
                 value: 'true'
               - name: DD_SOURCE
                 value: 'nodejs'
               - name: DD_HEALTH_PORT
                 value: '12345'
             image: gcr.io/datadoghq/serverless-init:<YOUR_TAG>
             name: serverless-init-1
             resources:
               limits:
                 cpu: 1000m
                 memory: 512Mi
             startupProbe:
               failureThreshold: 3
               periodSeconds: 10
               tcpSocket:
                 port: 12345
               timeoutSeconds: 1
             volumeMounts:
               - mountPath: /shared-volume
                 name: shared-volume
         volumes:
           - emptyDir:
               medium: Memory
               sizeLimit: 512Mi
             name: shared-volume
     traffic:
       - latestRevision: true
         percent: 100
   

   This example uses the latest version of serverless-init. See the latest releases on Docker Hub to pin a specific version.

   See the Environment Variables for more information.

   In this example, the environment variables, startup health check, and volume mount are already added. If you don’t want to enable logs, remove the shared volume.

   Ensure the container port for the main container is the same as the one exposed in your Dockerfile/service.

   To deploy your container, run:

   gcloud run services replace <FILENAME>.yaml
   

   After deploying your Cloud Run app, you can manually modify your app’s settings to enable Datadog monitoring.

   1. Create a Volume with In-Memory volume type.

   2. Add a new container with image URL: gcr.io/datadoghq/serverless-init:<YOUR_TAG>. See the latest releases on Docker Hub to pin a specific version.

   3. Add the volume mount to every container in your application. Choose a path such as /shared-volume, and remember it for the next step.

   4. Add the following environment variables to your serverless-init sidecar container:

      • DD_SERVICE: A name for your service. For example, gcr-sidecar-test.
      • DD_ENV: A name for your environment. For example, dev.
      • DD_SERVERLESS_LOG_PATH: Your log path. For example, /shared-volume/logs/*.log. The path must begin with the mount path you defined in the previous step.
      • DD_API_KEY: Your Datadog API key.

      For a list of all environment variables, including additional tags, see Environment variables.

3. Set up logs.

   In the previous step, you created a shared volume. You may have also set the DD_SERVERLESS_LOG_PATH environment variable, which defaults to /shared-volume/logs/app.log.

   In this step, configure your logging library to write logs to the file set in DD_SERVERLESS_LOG_PATH. In Node.js, we recommend writing logs in a JSON format. For example, you can use a third-party logging library such as winston:

   Copy

   const tracer = require('dd-trace').init({
     logInjection: true,
   });
   const { createLogger, format, transports } = require('winston');
   
   const LOG_FILE = "/shared-volume/logs/app.log"
   
   const logger = createLogger({
     level: 'info',
     exitOnError: false,
     format: format.json(),
     transports: [
       new transports.File({ filename: LOG_FILE }),
       new transports.Console()
     ],
   });
   
   logger.info('Hello world!');

   Datadog recommends setting the environment variables DD_LOGS_INJECTION=true (in your main container) and DD_SOURCE=nodejs (in your sidecar container) to enable advanced Datadog log parsing.

   For more information, see Correlating Node.js Logs and Traces.

4. Add a service label in Google Cloud. In your Cloud Run service’s info panel, add a label with the following key and value:

   Key	Value
   service	The name of your service. Matches the value provided as the DD_SERVICE environment variable.

   See Configure labels for services in the Cloud Run documentation for instructions.

5. Send custom metrics.

   To send custom metrics, view code examples. In serverless, only the distribution metric type is supported.

Environment variables

Distributed tracing with Pub/Sub

To get end-to-end distributed traces between Pub/Sub producers and Cloud Run services, configure your push subscriptions with the --push-no-wrapper and --push-no-wrapper-write-metadata flags. This moves message attributes from the JSON body to HTTP headers, allowing Datadog to extract producer trace context and create proper span links.

For more information, see Producer-aware tracing for Google Cloud Pub/Sub and Cloud Run and Payload unwrapping in the Google Cloud documentation.

Configure push subscriptions for full trace visibility

Create a new push subscription:

gcloud pubsub subscriptions create order-processor-sub \
  --topic=orders \
  --push-endpoint=https://order-processor-xyz.run.app/pubsub \
  --push-no-wrapper \
  --push-no-wrapper-write-metadata

Update an existing push subscription:

gcloud pubsub subscriptions update order-processor-sub \
  --push-no-wrapper \
  --push-no-wrapper-write-metadata

Configure Eventarc Pub/Sub triggers

Eventarc Pub/Sub triggers use push subscriptions as the underlying delivery mechanism. When you create an Eventarc trigger, GCP automatically creates a managed push subscription. However, Eventarc does not expose --push-no-wrapper-write-metadata as a trigger creation parameter, so you must manually update the auto-created subscription.

1. Create the Eventarc trigger:

   Copy

   gcloud eventarc triggers create order-processor-trigger \
     --destination-run-service=order-processor \
     --destination-run-region=us-central1 \
     --event-filters="type=google.cloud.pubsub.topic.v1.messagePublished" \
     --event-filters="topic=projects/my-project/topics/orders" \
     --location=us-central1

2. Find the auto-created subscription:

   Copy

   gcloud pubsub subscriptions list \
     --filter="topic:projects/my-project/topics/orders" \
     --format="table(name,pushConfig.pushEndpoint)"

   Example output:

   NAME                                                          PUSH_ENDPOINT
   eventarc-us-central1-order-processor-trigger-abc-sub-def      https://order-processor-xyz.run.app
   

3. Update the subscription for trace propagation:

   Copy

   gcloud pubsub subscriptions update \
     eventarc-us-central1-order-processor-trigger-abc-sub-def \
     --push-no-wrapper \
     --push-no-wrapper-write-metadata

Troubleshooting

This integration depends on your runtime having a full SSL implementation. If you are using a slim image, you may need to add the following command to your Dockerfile to include certificates:

RUN apt-get update && apt-get install -y ca-certificates

To have your Cloud Run services appear in the software catalog, you must set the DD_SERVICE, DD_VERSION, and DD_ENV environment variables.

Further reading

추가 유용한 문서, 링크 및 기사:

Tracing Node.js ApplicationsDOCUMENTATION Correlating Node.js Logs and TracesDOCUMENTATION