--- title: Autodiscovery with JMX description: >- Configure JMX-based integrations for containerized Java applications using Autodiscovery templates breadcrumbs: Docs > Containers > Containers Guides > Autodiscovery with JMX --- # Autodiscovery with JMX In containerized environments there are a few differences in how the Agent connects to the JMX server. Autodiscovery features make it possible to dynamically setup these integrations. Use Datadog's JMX based integrations to collect JMX applications metrics from your pods in Kubernetes. If you are using the Java tracer for your applications, you can alternatively take advantage of the [Java runtime metrics](https://docs.datadoghq.com/tracing/metrics/runtime_metrics/java/) feature to send these metrics to the Agent. ## Installation{% #installation %} ### Use a JMX-enabled Agent{% #use-a-jmx-enabled-agent %} JMX utilities are not installed in the Agent by default. To set up a JMX integration, append `-jmx` to your Agent's image tag. For example, `registry.datadoghq.com/agent:latest-jmx`. If you are using Datadog Operator or Helm, the following configurations append `-jmx` to your Agent's image tag: {% tab title="Operator" %} ```yaml apiVersion: datadoghq.com/v2alpha1 kind: DatadogAgent metadata: name: datadog spec: #(...) override: nodeAgent: image: jmxEnabled: true ``` {% /tab %} {% tab title="Helm" %} ```yaml agents: image: tagSuffix: jmx ``` {% /tab %} ## Configuration{% #configuration %} Use one of the following methods: - Autodiscovery annotations (recommended) - Autodiscovery configuration files: for heavy customization of configuration parameters ### Autodiscovery annotations{% #autodiscovery-annotations %} In this method, a JMX check configuration is applied using annotations on your Java-based Pods. This allows the Agent to automatically configure the JMX check when a new container starts. Ensure these annotations are on the created Pod, and not on the object (Deployment, DaemonSet, etc.) creating the Pod. Use the following template for Autodiscovery annotations: ```yaml apiVersion: v1 kind: Pod metadata: name: annotations: ad.datadoghq.com/.checks: | { "": { "init_config": { "is_jmx": true, "collect_default_metrics": true }, "instances": [{ "host": "%%host%%", "port": "" }] } } # (...) spec: containers: - name: '' # (...) env: - name: POD_IP valueFrom: fieldRef: fieldPath: status.podIP - name: JAVA_OPTS value: >- -Dcom.sun.management.jmxremote -Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.ssl=false -Dcom.sun.management.jmxremote.local.only=false -Dcom.sun.management.jmxremote.port= -Dcom.sun.management.jmxremote.rmi.port= -Djava.rmi.server.hostname=$(POD_IP) ``` In this example: - `` is the name of your pod. - `` matches the desired container within your pod. - `` is the name of the desired JMX integration. See the list of available JMX integrations. - Set `` as desired, as long as it matches between the annotations and `JAVA_OPTS`. With this configuration, the Datadog Agent discovers this pod and makes a request to the JMX server relative to the `%%host%%` [Autodiscovery template variable](https://docs.datadoghq.com/containers/guide/template_variables/)—this request resolves to the IP address of the discovered pod. This is why `java.rmi.server.hostname` is set to the `POD_IP` address previously populated with the [Kubernetes downward API](https://kubernetes.io/docs/concepts/workloads/pods/downward-api/). **Note**: The `JAVA_OPTS` environment variable is commonly used in Java-based container images as a startup parameter (for example, `java $JAVA_OPTS -jar app.jar`). If you are using a custom application, or if your application does not follow this pattern, set these system properties manually. #### Example annotation: Tomcat{% #example-annotation-tomcat %} The following configuration runs the [Tomcat](https://docs.datadoghq.com/integrations/tomcat/) JMX integration against port `9012`: ```yaml apiVersion: v1 kind: Pod metadata: name: tomcat-test annotations: ad.datadoghq.com/tomcat.checks: | { "tomcat": { "init_config": { "is_jmx": true, "collect_default_metrics": true }, "instances": [{ "host": "%%host%%", "port": "9012" }] } } spec: containers: - name: tomcat image: tomcat:8.0 imagePullPolicy: Always ports: - name: jmx-metrics containerPort: 9012 env: - name: POD_IP valueFrom: fieldRef: fieldPath: status.podIP - name: JAVA_OPTS value: >- -Dcom.sun.management.jmxremote -Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.ssl=false -Dcom.sun.management.jmxremote.local.only=false -Dcom.sun.management.jmxremote.port=9012 -Dcom.sun.management.jmxremote.rmi.port=9012 -Djava.rmi.server.hostname=$(POD_IP) ``` #### Custom metric annotation template{% #custom-metric-annotation-template %} If you need to collect additional metrics from these integrations, add them to the `init_config` section: ```yaml ad.datadoghq.com/.checks: | { "": { "init_config": { "is_jmx": true, "collect_default_metrics": true, "conf": [{ "include": { "domain": "java.lang", "type": "OperatingSystem", "attribute": { "FreePhysicalMemorySize": { "metric_type": "gauge", "alias": "jvm.free_physical_memory" } } } }] }, "instances": [{ "host": "%%host%%", "port": "" }] } } ``` See the [JMX integration](https://docs.datadoghq.com/integrations/java/) documentation for more information about the formatting for these metrics. ### Autodiscovery configuration files{% #autodiscovery-configuration-files %} If you need to pass a more complex custom configuration for your Datadog-JMX integration, you can use [Autodiscovery Container Identifiers](https://docs.datadoghq.com/containers/guide/ad_identifiers/?tab=kubernetes) to pass custom integration configuration files as well as a custom `metrics.yaml` file. #### 1. Compose configuration file When using this method, the Agent needs a configuration file and an optional `metrics.yaml` file for the metrics to collect. These files can either be mounted into the Agent pod or built into the container image. The configuration file naming convention is to first identify your desired integration name from the prerequisite steps of available integrations. Once this is determined, the Agent needs a configuration file named relative to that integration—*or* within that integration's config directory. For example, for the [Tomcat](https://docs.datadoghq.com/integrations/tomcat/) integration, create *either*: - `/etc/datadog-agent/conf.d/tomcat.yaml`, or - `/etc/datadog-agent/conf.d/tomcat.d/conf.yaml` If you are using a custom `metrics.yaml` file, include it in the integration's config directory. (For example: `/etc/datadog-agent/conf.d/tomcat.d/metrics.yaml`.) This configuration file should include `ad_identifiers`: ```yaml ad_identifiers: - init_config: is_jmx: true conf: instances: - host: "%%host%%" port: "" ``` Replace `` with the short image name of your desired container. For example, the container image `gcr.io/CompanyName/my-app:latest` has a short image name of `my-app`. As the Datadog Agent discovers that container, it sets up the JMX configuration as described in this file. You can alternatively reference and specify [custom identifiers to your containers](https://docs.datadoghq.com/containers/guide/ad_identifiers/?tab=kubernetes#custom-autodiscovery-container-identifiers) if you do not want to base this on the short image name. Like Kubernetes annotations, configuration files can use [Autodiscovery template variables](https://docs.datadoghq.com/containers/guide/template_variables/). In this case, the `host` configuration uses `%%host%%` to resolve to the IP address of the discovered container. See the [JMX integration](https://docs.datadoghq.com/integrations/java/) documentation (as well as the example configurations for the pre-provided integrations) for more information about structuring your `init_config` and `instances` configuration for the ``. #### 2. Mount configuration file {% tab title="Operator" %} If you are using Datadog Operator, add an override: ```yaml apiVersion: datadoghq.com/v2alpha1 kind: DatadogAgent metadata: name: datadog spec: #(...) override: nodeAgent: image: jmxEnabled: true extraConfd: configDataMap: .yaml: |- ad_identifiers: - init_config: is_jmx: true instances: - host: "%%host%%" port: "" ``` {% /tab %} {% tab title="Helm" %} In Helm, use the `datadog.confd` option: ```yaml datadog: confd: .yaml: | ad_identifiers: - init_config: is_jmx: true instances: - host: "%%host%%" port: "" ``` {% /tab %} {% tab title="Custom image" %} If you cannot mount these files in the Agent container (for example, on Amazon ECS) you can build an Agent Docker image containing the desired configuration files. For example: ```Dockerfile FROM gcr.io/datadoghq/agent:latest-jmx COPY conf.d/tomcat.d/ COPY conf.d/tomcat.d/ ``` Then use this new custom image as your regular containerized Agent. {% /tab %} #### 3. Expose JMX server Set up the JMX server in a way that allows the Agent to access it: ```yaml spec: containers: - # (...) env: - name: POD_IP valueFrom: fieldRef: fieldPath: status.podIP - name: JAVA_OPTS value: >- -Dcom.sun.management.jmxremote -Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.ssl=false -Dcom.sun.management.jmxremote.local.only=false -Dcom.sun.management.jmxremote.port= -Dcom.sun.management.jmxremote.rmi.port= -Djava.rmi.server.hostname=$(POD_IP) ``` ## Secure the JMX port{% #secure-the-jmx-port %} The configuration examples above disable JMX authentication and SSL (`-Dcom.sun.management.jmxremote.authenticate=false` and `-Dcom.sun.management.jmxremote.ssl=false`). Without additional safeguards, any pod in the cluster can connect to the JMX port and issue remote method invocations. To reduce this risk, use one or both of the following approaches. ### Restrict network access with a Kubernetes NetworkPolicy{% #restrict-network-access-with-a-kubernetes-networkpolicy %} Apply a [NetworkPolicy](https://kubernetes.io/docs/concepts/services-networking/network-policies/) that allows ingress on the JMX port only from the Datadog Agent pods. Apply this approach without changing your JMX or Agent configuration. It is compatible with any CNI plugin that supports NetworkPolicy (for example, Calico or Cilium). The Datadog Agent DaemonSet applies the label `app.kubernetes.io/component: agent` to its pods. Use this label as the ingress source selector: ```yaml apiVersion: networking.k8s.io/v1 kind: NetworkPolicy metadata: name: restrict-jmx-to-datadog-agent namespace: spec: podSelector: matchLabels: app: policyTypes: - Ingress ingress: - from: - podSelector: matchLabels: app.kubernetes.io/component: agent ports: - protocol: TCP port: ``` Replace `` with the namespace of your Java application, `` with the label that identifies your Java pods, and `` with the JMX port you configured. If the Datadog Agent runs in a different namespace from your application, add a `namespaceSelector` to the ingress rule: ```yaml - from: - namespaceSelector: matchLabels: kubernetes.io/metadata.name: podSelector: matchLabels: app.kubernetes.io/component: agent ports: - protocol: TCP port: ``` Replace `` with the namespace where the Datadog Agent is deployed. ### Enable JMX password authentication{% #enable-jmx-password-authentication %} To require credentials for JMX connections (an additional layer of defense), you can enable JMX authentication. The Datadog Agent JMX check supports `user` and `password` parameters. #### Step 1 - Create a Kubernetes Secret with JMX credentials{% #step-1---create-a-kubernetes-secret-with-jmx-credentials %} Create a JMX password file and access file, then store them in a Kubernetes Secret: ```bash # Create the password file (format: ) echo 'monitorRole ' > jmxremote.password chmod 400 jmxremote.password # Create the access file (format: readonly|readwrite) echo 'monitorRole readonly' > jmxremote.access chmod 400 jmxremote.access # Create the Kubernetes Secret kubectl create secret generic jmx-credentials \ --from-file=jmxremote.password \ --from-file=jmxremote.access \ -n ``` #### Step 2 - Configure the Java application{% #step-2---configure-the-java-application %} Mount the Secret and update the JVM flags to enable authentication: ```yaml spec: containers: - name: '' env: - name: POD_IP valueFrom: fieldRef: fieldPath: status.podIP - name: JAVA_OPTS value: >- -Dcom.sun.management.jmxremote -Dcom.sun.management.jmxremote.authenticate=true -Dcom.sun.management.jmxremote.password.file=/etc/jmx/jmxremote.password -Dcom.sun.management.jmxremote.access.file=/etc/jmx/jmxremote.access -Dcom.sun.management.jmxremote.ssl=false -Dcom.sun.management.jmxremote.local.only=false -Dcom.sun.management.jmxremote.port= -Dcom.sun.management.jmxremote.rmi.port= -Djava.rmi.server.hostname=$(POD_IP) volumeMounts: - name: jmx-credentials mountPath: /etc/jmx readOnly: true volumes: - name: jmx-credentials secret: secretName: jmx-credentials defaultMode: 0400 ``` #### Step 3 - Pass credentials in the Autodiscovery annotation{% #step-3---pass-credentials-in-the-autodiscovery-annotation %} Add `user` and `password` to the JMX check instance. Use the [`%%env_%%` template variable](https://docs.datadoghq.com/containers/guide/template_variables/) to reference the password from an environment variable in the Agent container rather than hardcoding it in the annotation: ```yaml ad.datadoghq.com/.checks: | { "": { "init_config": { "is_jmx": true, "collect_default_metrics": true }, "instances": [{ "host": "%%host%%", "port": "", "user": "monitorRole", "password": "%%env_JMX_PASSWORD%%" }] } } ``` Make `JMX_PASSWORD` available in the Agent container by injecting it from the same Kubernetes Secret: ```yaml # In your Datadog Agent DaemonSet or Helm values env: - name: JMX_PASSWORD valueFrom: secretKeyRef: name: jmx-credentials key: jmxremote.password ``` For the full list of security-related parameters, including SSL and client certificate options (`trust_store_path`, `key_store_path`, `rmi_registry_ssl`), see the [JMX integration](https://docs.datadoghq.com/integrations/java/) configuration reference. ## Available JMX integrations{% #available-jmx-integrations %} The Datadog Agent comes with several JMX integrations pre-configured. | Integration Name | Metrics file | Configuration file | | --------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------- | | [activemq](https://docs.datadoghq.com/integrations/activemq/) | [metrics.yaml](https://github.com/DataDog/integrations-core/blob/master/activemq/datadog_checks/activemq/data/metrics.yaml) | [conf.yaml.example](https://github.com/DataDog/integrations-core/blob/master/activemq/datadog_checks/activemq/data/conf.yaml.example) | | [cassandra](https://docs.datadoghq.com/integrations/cassandra/) | [metrics.yaml](https://github.com/DataDog/integrations-core/blob/master/cassandra/datadog_checks/cassandra/data/metrics.yaml) | [conf.yaml.example](https://github.com/DataDog/integrations-core/blob/master/cassandra/datadog_checks/cassandra/data/conf.yaml.example) | | [confluent_platform](https://docs.datadoghq.com/integrations/confluent_platform/) | [metrics.yaml](https://github.com/DataDog/integrations-core/blob/master/confluent_platform/datadog_checks/confluent_platform/data/metrics.yaml) | [conf.yaml.example](https://github.com/DataDog/integrations-core/blob/master/confluent_platform/datadog_checks/confluent_platform/data/conf.yaml.example) | | [hazelcast](https://docs.datadoghq.com/integrations/hazelcast/) | [metrics.yaml](https://github.com/DataDog/integrations-core/blob/master/hazelcast/datadog_checks/hazelcast/data/metrics.yaml) | [conf.yaml.example](https://github.com/DataDog/integrations-core/blob/master/hazelcast/datadog_checks/hazelcast/data/conf.yaml.example) | | [hive](https://docs.datadoghq.com/integrations/hive/) | [metrics.yaml](https://github.com/DataDog/integrations-core/blob/master/hive/datadog_checks/hive/data/metrics.yaml) | [conf.yaml.example](https://github.com/DataDog/integrations-core/blob/master/hive/datadog_checks/hive/data/conf.yaml.example) | | [hivemq](https://docs.datadoghq.com/integrations/hivemq/) | [metrics.yaml](https://github.com/DataDog/integrations-core/blob/master/hivemq/datadog_checks/hivemq/data/metrics.yaml) | [conf.yaml.example](https://github.com/DataDog/integrations-core/blob/master/hivemq/datadog_checks/hivemq/data/conf.yaml.example) | | [hudi](https://docs.datadoghq.com/integrations/hudi/) | [metrics.yaml](https://github.com/DataDog/integrations-core/blob/master/hudi/datadog_checks/hudi/data/metrics.yaml) | [conf.yaml.example](https://github.com/DataDog/integrations-core/blob/master/hudi/datadog_checks/hudi/data/conf.yaml.example) | | [ignite](https://docs.datadoghq.com/integrations/ignite/) | [metrics.yaml](https://github.com/DataDog/integrations-core/blob/master/ignite/datadog_checks/ignite/data/metrics.yaml) | [conf.yaml.example](https://github.com/DataDog/integrations-core/blob/master/ignite/datadog_checks/ignite/data/conf.yaml.example) | | [jboss_wildfly](https://docs.datadoghq.com/integrations/jboss_wildfly/) | [metrics.yaml](https://github.com/DataDog/integrations-core/blob/master/jboss_wildfly/datadog_checks/jboss_wildfly/data/metrics.yaml) | [conf.yaml.example](https://github.com/DataDog/integrations-core/blob/master/jboss_wildfly/datadog_checks/jboss_wildfly/data/conf.yaml.example) | | [kafka](https://docs.datadoghq.com/integrations/kafka/) | [metrics.yaml](https://github.com/DataDog/integrations-core/blob/master/kafka/datadog_checks/kafka/data/metrics.yaml) | [conf.yaml.example](https://github.com/DataDog/integrations-core/blob/master/kafka/datadog_checks/kafka/data/conf.yaml.example) | | [presto](https://docs.datadoghq.com/integrations/presto/) | [metrics.yaml](https://github.com/DataDog/integrations-core/blob/master/presto/datadog_checks/presto/data/metrics.yaml) | [conf.yaml.example](https://github.com/DataDog/integrations-core/blob/master/presto/datadog_checks/presto/data/conf.yaml.example) | | [solr](https://docs.datadoghq.com/integrations/solr/) | [metrics.yaml](https://github.com/DataDog/integrations-core/blob/master/solr/datadog_checks/solr/data/metrics.yaml) | [conf.yaml.example](https://github.com/DataDog/integrations-core/blob/master/solr/datadog_checks/solr/data/conf.yaml.example) | | [sonarqube](https://docs.datadoghq.com/integrations/sonarqube/) | [metrics.yaml](https://github.com/DataDog/integrations-core/blob/master/sonarqube/datadog_checks/sonarqube/data/metrics.yaml) | [conf.yaml.example](https://github.com/DataDog/integrations-core/blob/master/sonarqube/datadog_checks/sonarqube/data/conf.yaml.example) | | [tomcat](https://docs.datadoghq.com/integrations/tomcat/) | [metrics.yaml](https://github.com/DataDog/integrations-core/blob/master/tomcat/datadog_checks/tomcat/data/metrics.yaml) | [conf.yaml.example](https://github.com/DataDog/integrations-core/blob/master/tomcat/datadog_checks/tomcat/data/conf.yaml.example) | | [weblogic](https://docs.datadoghq.com/integrations/weblogic/) | [metrics.yaml](https://github.com/DataDog/integrations-core/blob/master/weblogic/datadog_checks/weblogic/data/metrics.yaml) | [conf.yaml.example](https://github.com/DataDog/integrations-core/blob/master/weblogic/datadog_checks/weblogic/data/conf.yaml.example) | Each integration in the above table has a `metrics.yaml` file predefined to match the expected pattern of the returned JMX metrics per application. Use the listed integration names as `` in your Autodiscovery annotations or configuration files. Alternatively use `jmx` as your `` to set up a basic JMX integration and collect the default `jvm.*` metrics only. ## Further Reading{% #further-reading %} - [Create and load an Autodiscovery integration template](https://docs.datadoghq.com/agent/kubernetes/integrations/) - [Match a container with the corresponding integration template](https://docs.datadoghq.com/agent/guide/ad_identifiers/) - [Manage which container to include in the Agent Autodiscovery](https://docs.datadoghq.com/agent/guide/autodiscovery-management/) - [Dynamically assign and collect tags from your application](https://docs.datadoghq.com/agent/kubernetes/tag/)