Private locations allow you to monitor internal-facing applications or any private URLs that aren’t accessible from the public internet. They can also be used to create a new custom Synthetics location.
The private location worker is shipped as a Docker container, so it can run on a Linux based OS or Windows OS if the Docker engine is available on your host and can run in Linux containers mode.
By default, every second, your private location worker pulls your test configurations from Datadog’s servers using HTTPS, executes the test depending on the frequency defined in the configuration of the test, and returns the test results to Datadog’s servers.
Once you created a private location, configuring a Synthetics API test from a private location is completely identical to the one of Datadog managed locations.
Go in Synthetics -> Settings -> Private Locations and create a new private location:
Fill out the Location Details and click Save and Generate to generate the configuration file associated with your private location on your worker.
Note: The configuration file contains secrets for private location authentication, test configuration decryption, and test result encryption. Datadog does not store the secrets, so store them locally before leaving the Private Locations screen. You need to be able to reference these secrets again if you decide to add more workers, or to install workers on another host.
Launch your worker as a standalone container using the Docker run command provided and the previously created configuration file:
docker run --init --rm -v $PWD/worker-config-<LOCATION_ID>.json:/etc/datadog/synthetics-check-runner.json datadog/synthetics-private-location-worker
One worker can process up to 10 tests in parallel by default. To scale a private location:
concurrencyparameter value to allow more parallel tests from one worker.
Nrequests depending on its number of free slots and when worker 1 is processing tests, worker 2 requests the following tests, etc.
To pull test configurations and push test results, the private location worker needs access to one of the Datadog API endpoints:
Check if the endpoint corresponding to your Datadog Site is available from the host runing the worker:
Note: You must allow outbound traffic on port
443 because test configurations are pulled and test results are pushed via HTTPS.
If your private location reports correctly to Datadog you should see the corresponding health status displayed if the private location polled your endpoint less than 5 seconds before loading the settings or create test pages:
You should now be able to use your new private location as any other Datadog managed locations for your Synthetics API tests.
synthetics-private-location-worker comes with a number of options that can be set to configure your private locations through the launch command or the configuration file. Arguments set in the launch command have precedence over the configuration file. However, these options aren’t stored and are consequently only prevalent for a given launch:
|Array of Strings||DNS server IPs used in given order (|
|Boolean||Use local DNS config in addition to –dnsServer (currently |
|Array of Strings||IANA IPv4/IPv6 Special-Purpose Address Registry||Deny access to IP ranges (e.g. |
|Array of Strings||Grant access to IP ranges (has precedence over |
|String||Datadog site (|
|String||Format log output [choices: |
|Integer||Maximum number of tests executed in parallel.|
|Integer||Maximum test execution duration, in milliseconds.|
|Integer||Maximum HTTP body size for download, in bytes.|
|Integer||Maximum HTTP body size for the assertions, in bytes.|
|Integer||Maximum duration for regex execution, in milliseconds.|
Note: These options and more can be found by running the help command for the Datadog worker
docker run --rm datadog/synthetics-private-location-worker --help.
If the traffic has to go through a proxy, you need to set the
proxy option to your proxy URL in a curl-like way (
--proxy=http://<YOUR_USER>:<YOUR_PWD>@<YOUR_IP>:<YOUR_PORT> URL for instance). If you use this, no additional configuration on your proxy should be needed.
By default, the Datadog workers use
184.108.40.206 to perform DNS resolution. If it fails, it makes a second attempt to communicate with
If you are testing an internal URL and need to use an internal DNS server you can set the
dnsServer option to a specific DNS IP address. Alternatively leverage the
dnsUseHost parameter to have your worker use your local DNS config from the
If you are using private locations to monitor internal endpoints, some of your servers might be using special-purpose IPv4. These IPs are blacklisted by default, so if your private location needs to run a test on one of them, you first need to whitelist it using the
The private location workers only pull data from Datadog servers. Datadog does not push data to the workers. The secret access key, used to authenticate your private location worker to the Datadog servers, uses an in-house protocol based on AWS Signature Version 4 protocol.
The test configurations are encrypted asymmetrically. The private key is used to decrypt the test configurations pulled by the workers from Datadog servers. The public key is used to encrypt the test results that are sent from the workers to Datadog’s servers.