This buildpack installs the Datadog Agent in your Heroku Dyno to collect system metrics, custom application metrics, and traces. To collect custom application metrics or traces, include the language appropriate DogStatsD or Datadog APM library in your application.
To add this buildpack to your project, as well as set the required environment variables:
cd <root of my project> # If this is a new Heroku project heroku create # Add the appropriate language-specific buildpack. For example: heroku buildpacks:add heroku/ruby # Enable Heroku Labs Dyno Metadata heroku labs:enable runtime-dyno-metadata -a $(heroku apps:info|grep ===|cut -d' ' -f2) # Add this buildpack and set your Datadog API key heroku buildpacks:add --index 1 https://github.com/DataDog/heroku-buildpack-datadog.git heroku config:add DD_API_KEY=<your API key> # Deploy to Heroku git push heroku master
Once complete, the Datadog Agent is started automatically when each Dyno starts.
The Datadog Agent provides a listening port on 8125 for StatsD/DogStatsD metrics and events. Traces are collected on port 8126.
In addition to the environment variables shown above, there are a number of others available:
||Required. Your API key is available from the Datadog API Integrations page. Note that this is the API key, not the application key.|
||Deprecated. WARNING: Setting the hostname manually may result in metrics continuity errors. It is recommended that you do not set this variable. Because dyno hosts are ephemeral it is recommended that you monitor based on the tags
||Optional. Set to
||Optional. Sets additional tags provided as a space-delimited string. For example,
||Optional. Optionally set additional percentiles for your histogram metrics. See How to graph percentiles.|
||Optional. When set, the Datadog Agent does not run.|
||Optional. Trace collection is enabled by default. Set this to
||Optional. By default, the buildpack installs the latest version of the Datadog Agent available in the package repository. Use this variable to install older versions of the Datadog Agent (note that not all versions of the Agent may be available).|
||Optional. The Datadog Agent automatically tries to identify your environment by searching for a tag in the form
Heroku dynos are ephemeral-they can move to different host machines whenever new code is deployed, configuration changes are made, or resouce needs/availability changes. This makes Heroku flexible and responsive, but can potentially lead to a high number of reported hosts in Datadog. Datadog bills on a per-host basis, and the buildpack default is to report actual hosts, which can lead to higher than expected costs.
Depending on your use case, you may want to set your hostname so that hosts are aggregated and report a lower number. To do this, Set
true. This causes the Agent to report the hostname as the dyno name (e.g.
run.1234), so your host count matches your dyno usage. One drawback is that you may see some metrics continuity errors whenever a dyno is cycled.
This log integration is currently in public beta
Heroku provides 3 types of logs:
App Logs: output from the application you pushed on the platform.
System Logs: messages about actions taken by the Heroku platform infrastructure on behalf of your app.
API Logs: administrative questions implemented by you and other developers working on your app.
Heroku’s HTTP/S drains buffer log messages and submit batches of messages to an HTTPS endpoint via a POST request.
The POST body contains Syslog formatted messages, framed using the Syslog TCP protocol octet counting framing method.
The Datadog HTTP API implements and understands the Logplex standard defined by the content-header
To send all these logs to Datadog:
heroku drains:add https://http-intake.logs.datadoghq.com/v1/input/<DD_API_KEY>?ddsource=heroku&service=<SERVICE>&host=<HOST> -a <APPLICATION_NAME>
heroku drains:add https://http-intake.logs.datadoghq.eu/v1/input/<DD_API_KEY>?ddsource=heroku&service=<SERVICE>&host=<HOST> -a <APPLICATION_NAME>
<DD_API_KEY>with your Datadog API Key.
<SERVICE>with your application name.
<HOST>with the desired hostname. Note: Per the host section, metrics and traces set the default host name to the dyno name. It is not yet possible to dynamically set the dyno name as the hostname for logs. For now, to correlate between metrics, traces, and logs the
dynotypetags can be used.
Add custom attributes to logs from your application by replacing the URL in the drain as follows:
Verify the Datadog Agent is listening by sending a custom metric:
# From your project directory: heroku run bash # Once your Dyno has started and you are at the command line echo -n "custom_metric:60|g|#shell" >/dev/udp/localhost/8125
After a few moments, use the metrics explorer to verify that the metric was received.
It can also be helpful to obtain Agent and Trace Agent logs from your running dyno.
Download Datadog Agent logs:
heroku ps:copy /app/.apt/var/log/datadog/datadog.log --dyno=<YOUR DYNO NAME>
Download Datadog Trace Agent logs:
heroku ps:copy /app/.apt/var/log/datadog/datadog-apm.log --dyno=<YOUR DYNO NAME>
Generate a flare by running:
# From your project directory: heroku run bash # Once your Dyno has started and you are at the command line, send a flare: agent -c /app/.apt/etc/datadog-agent/datadog.yaml flare
The flare contains the environment information and Datadog Agent configuration. However, since this is a new, stand-alone dyno, the logs are sparse and may not contain full log information.
Generate a more complete flare by setting your API key as an environment variable by running:
heroku ps:exec DD_API_KEY=<API KEY> agent -c /app/.apt/etc/datadog-agent/datadog.yaml flare
Visit the Github project page for more information and to view the source code.