Tutorial - Enabling Tracing for a Java Application with the Admission Controller

Overview

This tutorial walks you through the steps to enable tracing for Java Application using the Datadog Admission Controller.

For other scenarios, including on a host, in a container, on cloud infrastructure, and on applications written in other languages, see the other Enabling Tracing tutorials.

See Tracing Java Applications for general comprehensive tracing setup documentation for Java.

Prerequisites

Install the sample application

To demonstrate how to instrument your app with the Datadog Admission Controller, this tutorial uses a Java app built with Spring. You can find the code for the app in the springblog GitHub repository.

To get started, clone the repository:

git clone https://github.com/DataDog/springblog.git

The repository contains a multi-service Java application pre-configured to be run within Docker and Kubernetes. The sample app is a basic Spring app using REST.

Start and run the sample application

  1. Switch to to the /k8s subdirectory in the springblog repo:

    cd springblog/k8s/

  2. Deploy the workload with the depl.yaml file:

    kubectl apply -f ./depl.yaml

  3. Verify that it is running with the following command:

    kubectl get pods

    You should see something like this:

    NAME                                       READY   STATUS        RESTARTS        AGE
    springback-666db7b6b8-dzv7c                1/1     Terminating   0               2m41s
    springfront-797b78d6db-p5c84               1/1     Terminating   0               2m41s
    

    The service is started and listens on port 8080. It exposes an /upstream endpoint.

  4. Check that communication takes place by running the following curl command:

    curl localhost:8080/upstream
    Quote{type='success', values=Values{id=6, quote='Alea jacta est'}}

  5. To stop the application, run this command from the springblog/k8s directory so you can enable tracing on it:

    kubectl delete -f ./depl-with-lib-inj.yaml

Instrument your app with Datadog Admission Controller

After you have your application working, instrument it using the Datadog Admission Controller. In containerized environments, the process is generally:

  1. Install the Datadog Cluster Agent.
  2. Add Unified Service Tags in pod definition.
  3. Annotate your pod for library injection.
  4. Label your pod to instruct the Datadog Admission controller to mutate the pod.

There’s no need to add the tracing library because it’s automatically injected. You don’t need to redeploy your app yet. This section of the tutorial steps you through the process of adding Datadog variables and deploying a new image or version of your app.

  1. From the k8s subdirectory, use the following command to install the Datadog Cluster Agent, specifying the values-with-lib-inj.yaml config file and your Datadog API key:

    helm install datadog-agent -f values-with-lib-inj.yaml --set datadog.site='datadoghq.com' --set datadog.apiKey=$DD_API_KEY datadog/datadog

    For more detailed information, read Installing the Datadog Agent on Kubernetes with Helm
  2. You can check the Datadog Cluster Agent is running with the following command:

    kubectl get pods

    You should see something like this:

    NAME                                                READY   STATUS    RESTARTS  AGE
    datadog-agent-4s8rb                                 3/3     Running   0	        30s
    datadog-agent-cluster-agent-5666cffc44-d8qxk        1/1     Running   0         30s
    datadog-agent-kube-state-metrics-86f46b8484-mlqp7   1/1     Running   0         30s
    
  3. Add Unified Service Tags to the pod by adding the following block to the depl.yaml file:

    labels:
      tags.datadoghq.com/env: "dev"
      tags.datadoghq.com/service: "springfront"
      tags.datadoghq.com/version: "12"

  4. Configure the Datadog Admission Controller to inject a Java tracing library to the app container by adding the following annotation to the pod:

    annotations:
      admission.datadoghq.com/java-lib.version: "<CONTAINER IMAGE TAG>"

    Replace <CONTAINER IMAGE TAG> with the desired library version. Available versions are listed in the Java source repository

    Exercise caution when using the latest tag, as major library releases may introduce breaking changes.

    The final pod definition should look like the excerpt below. See also the full YAML file in the sample repo. The instructions you added to instrument the app are highlighted:

    apiVersion: apps/v1
    kind: Deployment
    metadata:
    name: springfront
    labels:
        tags.datadoghq.com/env: "dev"
        tags.datadoghq.com/service: "springfront"
        tags.datadoghq.com/version: "12"
    spec:
    replicas: 1
    selector:
        matchLabels:
        name: springfront
    minReadySeconds: 15
    strategy:
        type: RollingUpdate
        rollingUpdate:
        maxUnavailable: 1
        maxSurge: 1
    template:
        metadata:
        labels:
            name: springfront
            tags.datadoghq.com/env: "dev"
            tags.datadoghq.com/service: "springfront"
            tags.datadoghq.com/version: "12"
        annotations:
            admission.datadoghq.com/java-lib.version: "<CONTAINER IMAGE TAG>"
  5. Run the sample app with the following command:

    kubectl apply -f depl-with-lib-inj.yaml

  6. Run the following command to show that the app and Agent are running:

    kubectl get pods

    You should see something like this:

    NAME                                                READY   STATUS    RESTARTS   AGE
    datadog-agent-4s8rb                                 3/3     Running   0          28m
    datadog-agent-cluster-agent-5666cffc44-d8qxk        1/1     Running   0          28m
    datadog-agent-kube-state-metrics-86f46b8484-mlqp7   1/1     Running   0          28m
    springback-666db7b6b8-sb4tp                         1/1     Running   0          27m
    springfront-797b78d6db-mppbg                        1/1     Running   0          27m
    
  7. Run the following command to see details of the pod:

    kubectl describe pod springfront

    You should see something like this:

    Events:
    Type    Reason     Age   From               Message
    ----    ------     ----  ----               -------
    Normal  Scheduled  32s   default-scheduler  Successfully assigned default/springfront-797b78d6db-jqjdl to docker-desktop
    Normal  Pulling    31s   kubelet            Pulling image "gcr.io/datadoghq/dd-lib-java-init:latest"
    Normal  Pulled     25s   kubelet            Successfully pulled image "gcr.io/datadoghq/dd-lib-java-init:latest" in 5.656167878s
    Normal  Created    25s   kubelet            Created container datadog-lib-java-init
    Normal  Started    25s   kubelet            Started container datadog-lib-java-init
    Normal  Pulling    25s   kubelet            Pulling image "pejese/springfront:v2"
    Normal  Pulled     2s    kubelet            Successfully pulled image "pejese/springfront:v2" in 22.158699094s
    Normal  Created    2s    kubelet            Created container springfront
    Normal  Started    2s    kubelet            Started container springfront
    

    As you can see, an init-container is added to your pod. This container includes the Datadog Java tracing libraries to a volume mount. Also JAVA_TOOL_OPTIONS is modified to include javaagent. And Datadog-specific environment variables are added to the container:

    Environment:
    DD_ENV:              dev
    DD_VERSION:          12
    DD_SERVICE:          springfront
    DD_ENTITY_ID:         (v1:metadata.uid)
    DD_TRACE_AGENT_URL:  unix:///var/run/datadog/apm.socket
    URL:                 http://springback:8088
    JAVA_TOOL_OPTIONS:    -javaagent:/datadog-lib/dd-java-agent.jar
    Mounts:
    /datadog-lib from datadog-auto-instrumentation (rw)
    /var/run/datadog from datadog (rw)
    /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-qvmtk (ro)
    
  8. Verify that the Datadog tracing library is injected into the pod by checking the pod logs. For example:

    kubectl logs -f springfront-797b78d6db-jqjdl

    You should see something like this:

    Defaulted container "springfront" out of: springfront, datadog-lib-java-init (init)
    Picked up JAVA_TOOL_OPTIONS:  -javaagent:/datadog-lib/dd-java-agent.jar
    

View APM traces in Datadog

  1. Run the following command:

    curl localhost:8080/upstream

  2. Open the Datadog UI and see the two services reporting under the Service Catalog:

    Springback and springfront services in the Service Catalog.

  3. Explore Traces and see the associated Service Map:

    The flame graph that represents the service.
    The service map that represents the service.

Clean up the environment

Clean up your environment with the following command:

kubectl delete -f depl-with-lib-inj.yaml

Library injection with the Admission Controller simplifies service instrumentation, enabling you to view APM traces without changing or rebuilding your application. To learn more, read Datadog Library injection.

Troubleshooting

If you’re not receiving traces as expected, set up debug mode for the Java tracer. To learn more, read Enable debug mode.

Further reading