A sample application is available on GitHub.

Setup

  1. Install the Datadog Java tracer.

    1. Add the Datadog Java tracer to your Dockerfile:

      Dockerfile

      ADD 'https://dtdg.co/latest-java-tracer' agent.jar
      ENV JAVA_TOOL_OPTIONS="-javaagent:agent.jar"
    2. Add the tracer artifacts.

      <dependency>
        <groupId>com.datadoghq</groupId>
        <artifactId>dd-trace-api</artifactId>
        <version>DD_TRACE_JAVA_VERSION_HERE</version>
      </dependency>
      implementation 'com.datadoghq:dd-trace-api:DD_TRACE_JAVA_VERSION_HERE'

      See dd-trace-java releases for the latest tracer version.

    3. Add the @Trace annotation to any method you want to trace.

    For more information, see Tracing Java Applications.

  2. Install serverless-init as a sidecar.

    Setup

    Install the Datadog CLI client

    npm install -g @datadog/datadog-ci
    

    Install the gcloud CLI and authenticate with gcloud auth login.

    Configuration

    Configure the Datadog site and Datadog API key, and define the service name to use in Datadog.

    export DATADOG_SITE="<DATADOG_SITE>"
    export DD_API_KEY="<DD_API_KEY>"
    export DD_SERVICE="<SERVICE_NAME>"
    

    Instrument

    If you are new to Datadog serverless monitoring, launch the Datadog CLI in interactive mode to guide your first installation for a quick start.

    datadog-ci cloud-run instrument -i
    

    To permanently install Datadog for your production applications, run the instrument command in your CI/CD pipelines after your normal deployment. You can specify multiple services to instrument by passing multiple --service flags.

    datadog-ci cloud-run instrument --project <GCP-PROJECT-ID> --service <CLOUD-RUN-SERVICE-NAME> --region <GCP-REGION>
    

    You can pin to a specific image with the --sidecar-image flag. See the latest releases on Docker Hub.

    Additional parameters can be found in the CLI documentation.

    Create a YAML file that contains your configuration. You can use the following example and adapt it to your needs:

    apiVersion: serving.knative.dev/v1
    kind: Service
    metadata:
      name: '<SERVICE_NAME>'
      labels:
        cloud.googleapis.com/location: '<LOCATION>'
        service: '<SERVICE_NAME>'
    spec:
      template:
        metadata:
          labels:
            service: '<SERVICE_NAME>'
          annotations:
            # The maximum number of instances that can be created for this service.
            # https://cloud.google.com/run/docs/reference/rest/v1/RevisionTemplate
            autoscaling.knative.dev/maxScale: '100'
            # The startup CPU boost feature for revisions provides additional CPU during
            # instance startup time and for 10 seconds after the instance has started.
            # https://cloud.google.com/run/docs/configuring/services/cpu#startup-boost
            run.googleapis.com/startup-cpu-boost: 'true'
        spec:
          containers:
            - env:
                - name: DD_SERVICE
                  value: '<SERVICE_NAME>'
              image: '<CONTAINER_IMAGE>'
              name: run-sidecar-1
              ports:
                - containerPort: 8080
                  name: http1
              resources:
                limits:
                  cpu: 1000m
                  memory: 512Mi
              startupProbe:
                failureThreshold: 1
                periodSeconds: 240
                tcpSocket:
                  port: 8080
                timeoutSeconds: 240
              volumeMounts:
                - mountPath: /shared-volume
                  name: shared-volume
            - env:
                - name: DD_SERVERLESS_LOG_PATH
                  value: shared-volume/logs/*.log
                - name: DD_SITE
                  value: '<DATADOG_SITE>'
                - name: DD_ENV
                  value: '<ENV>'
                - name: DD_API_KEY
                  value: '<API_KEY>'
                - name: DD_SERVICE
                  value: '<SERVICE_NAME>'
                - name: DD_VERSION
                  value: '<VERSION>'
                - name: DD_LOG_LEVEL
                  value: debug
                - name: DD_LOGS_INJECTION
                  value: 'true'
                - name: DD_SOURCE
                  value: 'java'
                - name: DD_HEALTH_PORT
                  value: '12345'
              image: gcr.io/datadoghq/serverless-init:latest
              name: serverless-init-1
              resources:
                limits:
                  cpu: 1000m
                  memory: 512Mi
              startupProbe:
                failureThreshold: 3
                periodSeconds: 10
                tcpSocket:
                  port: 12345
                timeoutSeconds: 1
              volumeMounts:
                - mountPath: /shared-volume
                  name: shared-volume
          volumes:
            - emptyDir:
                medium: Memory
                sizeLimit: 512Mi
              name: shared-volume
      traffic:
        - latestRevision: true
          percent: 100
    

    This example uses the latest version of serverless-init. See the latest releases on Docker Hub to pin a specific version.

    See the Environment Variables for more information.

    In this example, the environment variables, startup health check, and volume mount are already added. If you don’t want to enable logs, remove the shared volume.

    Ensure the container port for the main container is the same as the one exposed in your Dockerfile/service.

    To deploy your container, run:

    gcloud run services replace <FILENAME>.yaml
    

    After deploying your Cloud Run app, you can manually modify your app’s settings to enable Datadog monitoring.

    1. Create a Volume with In-Memory volume type.

    2. Add a new container with image URL: gcr.io/datadoghq/serverless-init:latest. See the latest releases on Docker Hub to pin a specific version.

    3. Add the volume mount to every container in your application. Choose a path such as /shared-volume, and remember it for the next step.

    4. Add the following environment variables to your serverless-init sidecar container:

      • DD_SERVICE: A name for your service. For example, gcr-sidecar-test.
      • DD_ENV: A name for your environment. For example, dev.
      • DD_SERVERLESS_LOG_PATH: Your log path. For example, /shared-volume/logs/*.log. The path must begin with the mount path you defined in the previous step.
      • DD_API_KEY: Your Datadog API key.

      For a list of all environment variables, including additional tags, see Environment variables.

  3. Set up logs.

    In the previous step, you created a shared volume. You may have also set the DD_SERVERLESS_LOG_PATH environment variable, which defaults to /shared-volume/logs/app.log.

    In this step, configure your logging library to write logs to the file set in DD_SERVERLESS_LOG_PATH. In Java, we recommend writing logs in a JSON format. For example, you can use a third-party logging library such as Log4j 2:

    private static final Logger logger = LogManager.getLogger(App.class);
    logger.info("Hello World!");

    resources/log4j2.xml

    <Configuration>
      <Appenders>
        <Console name="Console"><JsonLayout compact="true" eventEol="true" properties="true"/></Console>
        <File name="FileAppender" fileName="/shared-volume/logs/app.log">
          <JsonLayout compact="true" eventEol="true" properties="true"/>
        </File>
      </Appenders>
      <Loggers><Root level="info"><AppenderRef ref="FileAppender"/></Root></Loggers>
    </Configuration>

    Datadog recommends setting the environment variables DD_LOGS_INJECTION=true (in your main container) and DD_SOURCE=java (in your sidecar container) to enable advanced Datadog log parsing.

    For more information, see Correlating Java Logs and Traces.

  4. Add a service label in Google Cloud. In your Cloud Run service’s info panel, add a label with the following key and value:

    KeyValue
    serviceThe name of your service. Matches the value provided as the DD_SERVICE environment variable.

    See Configure labels for services in the Cloud Run documentation for instructions.

  1. Send custom metrics.

    To send custom metrics, install the DogStatsD client and view code examples. In serverless, only the distribution metric type is supported.

Environment variables

VariableDescriptionContainer
DD_API_KEYDatadog API key - RequiredSidecar container
DD_SITEDatadog site - RequiredSidecar container
DD_SERVICEDatadog Service name. RequiredBoth containers
DD_SERVERLESS_LOG_PATHThe path where the sidecar should tail logs from. Recommended to set to /shared-volume/logs/app.log.Sidecar container
DD_LOGS_INJECTIONWhen true, enrich all logs with trace data for supported loggers. See Correlate Logs and Traces for more information.Application container
DD_VERSIONSee Unified Service Tagging.Both containers
DD_ENVSee Unified Service Tagging.Both containers
DD_SOURCESet the log source to enable a Log Pipeline for advanced parsing. To automatically apply language-specific parsing rules, set to java, or use your custom pipeline. Defaults to cloudrun.Sidecar container
DD_TAGSAdd custom tags to your logs, metrics, and traces. Tags should be comma separated in key/value format (for example: key1:value1,key2:value2).Sidecar container
JAVA_TOOL_OPTIONSRequired for tracing. The path to the Datadog Java agent. For example, -javaagent:/path/to/dd-java-agent.jar.Application container

Troubleshooting

This integration depends on your runtime having a full SSL implementation. If you are using a slim image, you may need to add the following command to your Dockerfile to include certificates:

RUN apt-get update && apt-get install -y ca-certificates

To have your Cloud Run services appear in the software catalog, you must set the DD_SERVICE, DD_VERSION, and DD_ENV environment variables.

If you are missing logs or traces during container shutdown, specify a container start up order to make your main container depend on the sidecar container.

Further reading