Overview
The following describes how to configure Datadog to gather Metrics and Logs from JFrog Artifactory and JFrog Xray.
JFrog Artifactory and Xray
JFrog Enterprise with Xray features Artifactory Enterprise and Xray. Together, they empower DevOps teams to improve their productivity to increase velocity and deliver high quality software releases with confidence.
Artifactory supports multiple build packages, artifacts, and metadata. It allows DevOps teams to have the freedom of choice of build packages like Bower, Chef, CocoaPods, Conan, Conda, CRAN, Debian, Docker, Golang, Gradle, Git LFS, Helm, Ivy, Maven, npm, NuGet, Opkg, P2, PHP Composer, Puppet, PyPI, RPM, RubyGems, SBT, Vagrant & VCS, CI/CD platforms, and devops tools.
Artifactory Enterprise supports multi-region, multi-cloud, and hybrid replication for geographically distributed teams allowing you to replicate between a source repository and multiple targets simultaneously and security features such as IP filtering, CNAME, and data encryption at rest. Artifactory supports Kubernetes for microservices and containerized applications. Manage your deployments and gain insight into dependencies using Artifactory as your Kubernetes registry.
JFrog Xray is a continuous security and universal artifact analysis solution that provides multi-layered analysis of your containers and software artifacts for vulnerabilities and license compliance issues. It is the only Software Composition Analysis solution that natively integrates with JFrog Artifactory for optimized scanning and unified operation. Supports all major package types, understands how to unpack them, and uses recursive scanning to see into all of the underlying layers and dependencies, even those packaged in Docker images, and zip files.
JFrog Artifactory and Xray logs Datadog dashboard
The JFrog Datadog integration allows you to send Artifactory/Xray logs to the log stream in Datadog. You can use it to enhance your existing dashboards or to gain more insight into JFrog Artifactory’s usage statistics or JFrog Xray’s scanned components details.
JFrog Artifactory and Xray metrics API dashboard
JFrog Artifactory and Xray metrics API integration with Datadog allows you to send metrics from the OpenMetrics API endpoint to Datadog. With this integration, you can gain insights into the system performance, storage consumption, and connection statistics associated with JFrog Artifactory/Xray, as well as, insights into the count and type of artifacts and components scanned by Xray. After setting up the configuration, these metrics are available as out-of-the-box dashboards and may be used to enhance existing dashboards within Datadog.
Setup
Metrics collection
Note:
Metrics collection is available only for JFrog Platform Self-Hosted users. JFrog Cloud is not supported.
Enable Metrics for Artifactory and Xray:
- Enable Metrics for Artifactory
- Create admin access tokens for Artifactory and Xray
Datadog Configuration
Follow the instructions below to configure this check for an Agent running on a host. For containerized environments, see the Containerized section.
These values override the configuration specified below
ARTIFACTORY_HOST_NAME_OR_IP -> IP address or DNS of Artifactory
ARTIFACTORY_ADMIN_TOKEN -> Admin token for Artifactory
XRAY_ADMIN_TOKEN -> Admin token for Xray
To configure this check for an Agent running on a host:
- Edit the openmetrics.d/conf.yaml file at the root of your Agent’s configuration directory to start collecting your Artifactory and Xray Metrics. See the sample openmetrics.d/conf.yaml for all available configuration options
instances:
- prometheus_url: http://<ARTIFACTORY_HOST_NAME_OR_IP>:80/artifactory/api/v1/metrics
scheme: http
headers:
Authorization: "Bearer <ARTIFACTORY_ADMIN_TOKEN>"
static_configs:
- targets: ["<ARTIFACTORY_HOST_NAME_OR_IP>:80"]
namespace: jfrog.artifactory
metrics:
- sys*
- jfrt*
- app*
- prometheus_url: http://<ARTIFACTORY_HOST_NAME_OR_IP>:80/xray/api/v1/metrics
scheme: http
headers:
Authorization: "Bearer <XRAY_ADMIN_TOKEN>"
namespace: jfrog.xray
metrics:
- app*
- db*
- go*
- queue*
- sys*
- jfxr*
- Restart the Agent. For containerized environments, see the Autodiscovery Integration Templates for guidance on applying the parameters specified above. To validate that the changes are applied, run the Agent’s status subcommand and look for
openmetrics under the Checks section.
Log collection - Using FluentD
Requirements
Install Fluentd using the jFrog documentation based on your installation type, and define the environment variable.
Configure Fluentd with Artifactory by downloading the Artifactory Fluentd configuration file to a directory you have permissions to write in, such as $JF_PRODUCT_DATA_INTERNAL locations.
cd $JF_PRODUCT_DATA_INTERNAL
wget https://raw.githubusercontent.com/jfrog/log-analytics-datadog/master/fluent.conf.rt
Override the match directive (last section) of the downloaded fluent.conf.rt with the details given below:
<match jfrog.**>
@type datadog
@id datadog_agent_jfrog_artifactory
api_key API_KEY
include_tag_key true
dd_source fluentd
</match>
API_KEY (required) is the API key from Datadog.dd_source is the name of the log integration in your logs in order to trigger the integration automatic setup in datadog.include_tag_key defaults to false and adds the fluentd tag in the JSON record if set to true.
Configure Fluentd with Xray by downloading the Xray Fluentd configuration file to a directory you have permissions to write, such as the $JF_PRODUCT_DATA_INTERNAL locations.
cd $JF_PRODUCT_DATA_INTERNAL
wget https://raw.githubusercontent.com/jfrog/log-analytics-datadog/master/fluent.conf.xray
Fill in the JPD_URL, USER, JFROG_API_KEY fields in the source directive of the downloaded fluent.conf.xray with the details given below:
<source>
@type jfrog_siem
tag jfrog.xray.siem.vulnerabilities
jpd_url JPD_URL
username USER
apikey JFROG_API_KEY
pos_file "#{ENV['JF_PRODUCT_DATA_INTERNAL']}/log/jfrog_siem.log.pos"
</source>
JPD_URL (required) is the Artifactory JPD URL of the format http://<ip_address>, which is used to pull Xray Violations.USER (required) is the Artifactory username for authentication.JFROG_API_KEY (required) is the Artifactory API Key for authentication.
Override the match directive (last section) of the downloaded fluent.conf.xray with the details given below:
<match jfrog.**>
@type datadog
@id datadog_agent_jfrog_xray
api_key API_KEY
include_tag_key true
dd_source fluentd
</match>
API_KEY (required) is the API key from Datadog.dd_source is the name of the log integration in your logs in order to trigger the integration automatic setup in Datadog.include_tag_key defaults to false and adds the fluentd tag in the json record if set to true.
Enable the integration by running td-agent on artifactory and xray instances:
The API key is configured in td-agent, which starts sending logs to Datadog. For other types of installation, see the JFrog documentation.
Add all attributes as facets from Facets > Add (on the left side of the screen in Logs) > Search.
If you have not installed the JFrog platform tile yet, install the tile.
JFrog dashboards
Go to Dashboard -> Dashboard List, find JFrog Artifactory Dashboard, Artifactory Metrics, Xray Metrics, Xray Logs, Xray Violations and explore it.
Data Collected
Metrics
See metadata.csv for a list of metrics provided by this check.
Troubleshooting
Need help? Contact Datadog support.
