To refine your search to traffic between particular endpoints, aggregate and filter your network aggregate connections with tags. You can select tags for the source and destination by using the search bar at the top of the page.
The following screenshot shows the default view, which aggregates the source and destination by the service tag. Accordingly, each row in the table represents service-to-service aggregate connections when aggregated over a one hour time period.
The next example shows all aggregate connections from IP addresses representing services in region us-east-1 to availability zones:
You can set the timeframe over which traffic is aggregated using the time selector at the top right of the page:
Facet panels mirror the tags in your search bar query. Switch between the facet panels with the Source and Destination tabs on top:
Aggregate and filter your traffic data by any tags in Datadog network page. A whitelist of tags is provided by default, which you can find in the search bar dropdown menu:
Whitelisted tags include service, availability zone, env, environment, pod, host, ip, and port, among others. If you want to aggregate or filter traffic by a tag that is not already in the menu, add it as a custom Facet:
+ button on the top right of the facet panels.Create.Once the custom facet is created, use this tag to filter and aggregate traffic in the network page and map. All custom facets can be viewed in the bottom Custom section of the facet panels.
To perform a multi-character wildcard search, use the * symbol as follows:
service:web* matches all services that start with webservice:*web matches all services that end with webservice:*web* matches all services that contain the string webWildcard searches work within facets with this syntax. This query returns all the services that end with the string mongo:
service:*mongo
To learn more, see the search syntax documentation.
Your network metrics are displayed through the graphs and the associated table. All sent and received metrics are displayed from the perspective of the source :
Values displayed might be different for sent_metric(source to destination) and received_metric(destination to source) if there is a large number of packet drops. In this case, if the destination sends a lot of bytes to the source, the aggregate connections that originate at destination include those bytes, but the aggregate connections that originate at source do not see them as received.
Note: The default collection interval is five minutes and retention is seven days.
The following network load metrics are available:
| Metric | Description |
|---|---|
| Volume | The number of bytes sent or received over a period. Measured in bytes (or orders of magnitude thereof) bidirectional. |
| Throughput | The rate of bytes sent or received over a period. Measured in bytes per second, bidirectional. |
TCP is a connection-oriented protocol that guarantees in-order delivery of packets. The following TCP metrics are available:
| Metric | Description |
|---|---|
| TCP Retransmits | TCP Retransmits represent detected failures that are retransmitted to ensure delivery. Measured in count of retransmits from the source. |
| TCP Latency | Measured as TCP smoothed round-trip time, that is the time between a TCP frame being sent and acknowledged. |
| TCP Jitter | Measured as TCP smoothed round-trip time variance. |
| Established Connections | The number of TCP connections in an established state. Measured in connections per second from the source. |
| Closed Connections | The number of TCP connections in a closed state. Measured in connections per second from the source. |
Filtering by specific AWS cloud services can help pinpoint latency, assess database performance, and visualize your network more completely. For instance, you can filter a search query by service, view the service in the Network Map, and trace communication on that node to see affected services.
service:s3, service:kinesis, and service:elb. For some services, you can break down latency and retransmits further by using more out-of-the-box tags like s3_bucket and rds_instance_type.NPM automatically detects S3, RDS, Kinesis, ELB, Elasticache, and others listed in the supported services. To monitor other endpoints where an Agent cannot be installed (such as public APIs), group the destination in the Network Overview by the domain tag.
Starting with Agent 7.17+, the Agent resolves IPs to human-readable domain names for external and internal traffic. Domain allows you to monitor cloud provider endpoints where a Datadog Agent cannot be installed, such as S3 buckets, application load balancers, and APIs. Unrecognizable domain names such as DGA domains from C&C servers may point to network security threats. Domain is encoded as a tag in Datadog, so you can use it in search bar queries and the facet panel to aggregate and filter traffic.
Note: DNS resolution is supported for hosts where the system probe is running on the root network namespace, which is usually caused by running the system-probe in a container without using the host network.
The Network Address Translation (NAT) is a tool used by Kubernetes and other systems to route traffic between containers. When investigating a specific dependency (for example, service to service), you can use the presence or absence of pre-NAT IPs to distinguish between Kubernetes-native services, which do their own routing, and services that rely on external clients for routing. This feature does not currently include resolution of NAT gateways.
To view pre-NAT and post-NAT IPs, use the Show pre-NAT IPs toggle in the table settings. When this setting is toggled off, IPs shown in the Source IP and Dest IP columns are by default post-NAT IPs. In cases where you have multiple pre-NAT IPs for one post-NAT IP, the top 5 most common pre-NAT IPs will be displayed. pre_nat.ip is a tag like any other in the product, so you can use it to aggregate and filter traffic.
NPM users may configure their networks to have overlapping IP spaces. For instance, you may want to deploy in multiple VPCs (virtual private clouds) which have overlapping address ranges and communicate only through load balancers or cloud gateways.
To correctly classify traffic destinations, NPM uses the concept of a network ID, which is represented as a tag. A network ID is an alphanumeric identifier for a set of IP addresses that can communicate with one another. When an IP address mapping to several hosts with different network IDs is detected, this identifier is used to determine the particular host network traffic is going to or coming from.
In AWS and GCP, the network ID is automatically set to the VPC ID. For other environments, the network ID may be set manually, either in datadog.yaml as shown below, or by adding the DD_NETWORK_ID to the process and core Agent containers.
network:
Id: <your-network-id>
Organize and share views of traffic data. Saved Views make debugging faster and empower collaboration. For instance, you can create a view, save it for the future for common queries, and copy its link to share network data with your teammates.
To learn more, see the Saved Views documentation.
The network table breaks down the Volume, Throughput, TCP Retransmits, Round-trip Time (RTT), and RTT variance metrics between each source and destination defined by your query.
You can configure the columns in your table using the Customize button at the top right of the table.
Congifure the traffic shown with the Filter Traffic button.
External traffic (to public IPs) and Datadog Agent traffic is shown by default. To narrow down your view, you can choose to toggle off the Show Datadog Traffic and Show External Traffic toggles.
Unresolved source and destination tags are marked as N/A. A traffic source or destination endpoint may be unresolved because:
Use the Show N/A (Unresolved Traffic) toggle in the upper right corner of the data table to filter out aggregate connections with unresolved (N/A) sources or destinations.
Select any row from the data table to see associated logs, traces, and processes for a given source <=> destination aggregate connection:
The sidepanel provides contextual telemetry to help you debug network dependencies. Use the Flows, Logs, Traces, and Processes tabs to determine whether a high retransmit count or latency in traffic between two endpoints is due to:
The top of the sidepanel displays common source and destination tags shared by the inspected dependency’s most recent connections. Use common tags to gain additional context into a faulty endpoint. For instance, when troubleshooting latent communication to a particular service, common destination tags will surface:
Additional helpful documentation, links, and articles:
