Overview
gRPC tests allow you to proactively monitor your gRPC services and servers. You can choose from two types:
- Behavior Checks
- Send gRPC requests to your applications’ API endpoints to verify responses and defined conditions, such as overall response time, header, or body content.
- Health Checks
- gRPC health checks are a standard for reporting the health of gRPC services. Determine if your gRPC servers and services are responsive, running, and capable of handling remote procedure calls (RPCs).
By implementing gRPC health checks, you can run gRPC health checks tests without having to provide a .proto
file to Datadog. For more information, see the example health checks .proto
file shared by the gRPC community.
gRPC tests can run from both managed and private locations depending on your preference for running the test from outside or inside your network. gRPC tests can run on a schedule, on-demand, or directly within your CI/CD pipelines.
Configuration
After choosing to create a gRPC
test, define your test’s request.
Define request
Specify the Host and Port to run your test on. The default gRPC port is 50051
.
Select Behavior Check to perform a unary call or Health Check to perform a health check.
For a behavior check, specify the Server Reflection or upload a Proto File that defines your gRPC server. Select a method and include a request message. Datadog does not support streaming methods.
<div class="shortcode-wrapper shortcode-img expand"><figure class="text-center"><a href="https://datadog-docs.imgix.net/images/synthetics/api_tests/grpc_behavior_check_test.7351f719c503d4cee14c02c7cb56755c.png?fit=max&auto=format" class="pop" data-bs-toggle="modal" data-bs-target="#popupImageModal"><picture class="" style="width:90%;" >
<img
class="img-fluid"
srcset="https://datadog-docs.imgix.net/images/synthetics/api_tests/grpc_behavior_check_test.7351f719c503d4cee14c02c7cb56755c.png?auto=format"
style="width:90%;" alt="Define gRPC request" />
</picture></a></figure>
</div>
For a health check, enter the name of the service. Leave this field blank if you want to send a health check on the gRPC server.
<div class="shortcode-wrapper shortcode-img expand"><figure class="text-center"><a href="https://datadog-docs.imgix.net/images/synthetics/api_tests/grpc_health_check_test.e00072f3e0e5d2f92ef32a73d6c280d3.png?fit=max&auto=format" class="pop" data-bs-toggle="modal" data-bs-target="#popupImageModal"><picture class="" style="width:90%;" >
<img
class="img-fluid"
srcset="https://datadog-docs.imgix.net/images/synthetics/api_tests/grpc_health_check_test.e00072f3e0e5d2f92ef32a73d6c280d3.png?auto=format"
style="width:90%;" alt="Define gRPC request" />
</picture></a></figure>
</div>
Add Advanced Options (optional) to your test:
- Timeout: Specify the amount of time in seconds before the test times out.
- Ignore server certificate error: Select to have your gRPC test go on with connection even if there are errors when validating the SSL certificate.
- gRPC metadata: Add and define metadata to your gRPC request to pass metadata between services.
Client certificate: Authenticate through mTLS by uploading your client certificate (.crt
) and the associated private key (.key
) in PEM
format.
You can use the openssl
library to convert your certificates. For example, convert a PKCS12
certificate to PEM
formatted private keys and certificates.
openssl pkcs12 -in <CERT>.p12 -out <CERT_KEY>.key -nodes -nocerts
openssl pkcs12 -in <CERT>.p12 -out <CERT>.cert -nokeys
Name your gRPC test.
Add env
Tags as well as any other tag to your gRPC test. You can then use these tags to filter through your Synthetic tests on the Synthetic Monitoring & Continuous Testing page.
Click Send to try out the request configuration. A response preview is displayed on the right side of your screen.
Define assertions
Assertions define what an expected test result is. After you click Send, an assertion on the response time
is added based on the response that was obtained. You must define at least one assertion for your test to monitor.
Type | Operator | Value type |
---|
response time | is less than | Integer (ms) |
gRPC response | contains , does not contain , is , is not ,
matches , does not match ,
jsonpath , xpath | String Regex |
gRPC metadata | is , is not , contains , does not contain , matches regex , does not match regex , does not exist | Integer (ms) Regex |
You can create up to 20 assertions per API test by clicking New Assertion or by clicking directly on the response preview:
Type | Operator | Value type |
---|
response time | is less than | Integer (ms) |
healthcheck status | is , is not | Integer (ms) |
gRPC metadata | is , is not , contains , does not contain , matches regex , does not match regex , does not exist | Integer (ms) |
You can create up to 20 assertions per API test by clicking New Assertion or by clicking directly on the response preview:
If a test does not contain an assertion on the response body, the body payload drops and returns an associated response time for the request within the timeout limit set by the Synthetics Worker.
If a test contains an assertion on the response body and the timeout limit is reached, an Assertions on the body/response cannot be run beyond this limit
error appears.
Select locations
Select the Locations to run your gRPC test from. gRPC tests can run from both managed and private locations depending on your preference for running the test from outside or inside your network.
Datadog’s out-of-the-box managed locations allow you to test public-facing websites and endpoints from regions where your customers are located.
Americas | APAC | EMEA |
---|
Canada Central (AWS) | Hong Kong (AWS) | Cape Town (AWS) |
Northern California (AWS) | Mumbai (AWS) | Frankfurt (AWS) |
Northern Virginia (AWS) | Seoul (AWS) | Ireland (AWS) |
Ohio (AWS) | Singapore (AWS) | London (AWS) |
Oregon (AWS) | Sydney (AWS) | Paris (AWS) |
São Paulo (AWS) | Tokyo (AWS) | Stockholm (AWS) |
Virginia (Azure) | Osaka (AWS) | Milan (AWS) |
| Jakarta (AWS) | Bahrain (AWS) |
The Datadog for Government site (US1-FED) uses the following managed location:
Specify test frequency
gRPC tests can run:
- On a schedule to ensure your most important services are always accessible to your users. Select the frequency at which you want Datadog to run your gRPC test.
- Within your CI/CD pipelines to start shipping without fearing faulty code might impact your customers experience.
- On-demand to run your tests whenever makes the most sense for your team.
Define alert conditions
Set alert conditions to determine the circumstances under which you want a test to fail and trigger an alert.
Alerting rule
When you set the alert conditions to: An alert is triggered if any assertion fails for X minutes from any n of N locations
, an alert is triggered only if these two conditions are true:
- At least one location was in failure (at least one assertion failed) during the last X minutes;
- At one moment during the last X minutes, at least n locations were in failure.
Fast retry
Your test can trigger retries X
times after Y
ms in case of a failed test result. Customize the retry interval to suit your alerting sensibility.
Location uptime is computed on a per-evaluation basis (whether the last test result before evaluation was up or down). The total uptime is computed based on the configured alert conditions. Notifications sent are based on the total uptime.
A notification is sent by your test based on the alerting conditions previously defined. Use this section to define how and what to message your team.
Similar to how you configure monitors, select users and/or services that should receive notifications either by adding an @notification
to the message or by searching for team members and connected integrations with the dropdown menu.
Enter the notification message for your test. This field allows standard Markdown formatting and supports the following conditional variables:
Conditional Variable | Description |
---|
{{ #is_alert }} | Show when the test alerts. |
{{ ^is_alert }} | Show unless the test alerts. |
{{ #is_recovery }} | Show when the test recovers from alert. |
{{ ^is_recovery }} | Show unless the test recovers from alert. |
{{ #is_renotify }} | Show when the monitor renotifies. |
{{ ^is_renotify }} | Show unless the monitor renotifies. |
{{ #is_priority }} | Show when the monitor matches priority (P1 to P5). |
{{ ^is_priority }} | Show unless the monitor matches priority (P1 to P5). |
Specify how often you want your test to re-send the notification message in case of test failure. To prevent renotification on failing tests, leave the option as Never renotify if the monitor has not been resolved
.
Click Create to save your test configuration and monitor.
For more information, see Using Synthetic Test Monitors.
Variables
Create local variables
To create a local variable, click Create a Local Variable. You can select one of the following available builtins to add to your variable string:
- {{ numeric(n) }}
- Generates a numeric string with
n
digits. - {{ alphabetic(n) }}
- Generates an alphabetic string with
n
letters. - {{ alphanumeric(n) }}
- Generates an alphanumeric string with
n
characters. - {{ date(n unit, format) }}
- Generates a date in one of Datadog’s accepted formats with a value corresponding to the UTC date the test is initiated at + or -
n
units. - {{ timestamp(n, unit) }}
- Generates a timestamp in one of Datadog’s accepted units with a value corresponding to the UTC timestamp the test is initiated at +/-
n
units. - {{ uuid }}
- Generates a version 4 universally unique identifier (UUID).
- {{ public-id }}
- Injects the Public ID of your test.
- {{ result-id }}
- Injects the Result ID of your test run.
To obfuscate local variable values in test results, select Hide and obfuscate variable value. Once you have defined the variable string, click Add Variable.
Use variables
You can use the global variables defined on the Settings page in the URL, advanced options, and assertions of your gRPC tests.
To display your list of variables, type {{
in your desired field.
Test failure
A test is considered FAILED
if it does not satisfy one or more assertions or if the request prematurely failed. In some cases, the test can fail without testing the assertions against the endpoint.
These reasons include the following:
gRPC specific errors
- gRPC has a list of specific status codes that can be found in the official gRPC documentation.
CONNRESET
- The connection was abruptly closed by the remote server. Possible causes include the web server encountering an error or crashing while responding, or losing connectivity to the web server.
DNS
- DNS entry not found for the test URL. Possible causes include a misconfigured test URL or the wrong configuration of your DNS entries.
INVALID_REQUEST
- The configuration of the test is invalid (for example, a typo in the URL).
SSL
- The SSL connection couldn’t be performed. See the dedicated error page for more information.
TIMEOUT
- The request couldn’t be completed in a reasonable time. Two types of
TIMEOUT
can happen:TIMEOUT: The request couldn't be completed in a reasonable time.
indicates that the request duration hit the test defined timeout (default is set to 60 seconds).
For each request, only the completed stages for the request are displayed in the network waterfall. For example, in the case of Total response time
only being displayed, the timeout occurred during the DNS resolution.TIMEOUT: Overall test execution couldn't be completed in a reasonable time.
indicates that the test duration (request and assertions) hits the maximum duration of 60.5 seconds.
Permissions
By default, only users with the Datadog Admin and Datadog Standard roles can create, edit, and delete Synthetic gRPC tests. To get create, edit, and delete access to Synthetic gRPC tests, upgrade your user to one of those two default roles.
If you are using the custom role feature, add your user to any custom role that includes synthetics_read
and synthetics_write
permissions.
Restrict access
Access restriction is available for customers using custom roles on their accounts.
You can restrict access to a browser test based on the roles in your organization. When creating a browser test, choose which roles (in addition to your user) can read and write your test.
Further Reading
Additional helpful documentation, links, and articles: