Getting Started with the Agent
Tagging is used throughout the Datadog product to make it easier to subset and query the machines and metrics that you have to monitor. Without the ability to assign and filter based on tags, finding the problems that exist in your environment and narrowing them down enough to discover the true causes would be extremely difficult.
There are four primary ways to assign tags: inherited from the integration, in the configuration, in the UI, and using the API, though the UI and API only allow you to assign tags at the host level. The recommended method is to rely on the integration or via the configuration files.
The easiest method for assigning tags is to rely on the integration. Tags assigned to your Amazon Web Services instances, Chef recipes, Docker labels, and more are all automatically assigned to the hosts and metrics when they are brought in to Datadog.
The following integration sources create tags automatically in Datadog:
|Amazon EC2||AMI, Customer Gateway, DHCP Option, EBS Volume, Instance, Internet Gateway, Network ACL, Network Interface, Reserved Instance, Reserved Instance Listing, Route Table , Security Group - EC2 Classic, Security Group - VPC, Snapshot, Spot Batch, Spot Instance Request, Spot Instances, Subnet, Virtual Private Gateway, VPC, VPN Connection|
|Amazon Elastic File System||Filesystem|
|Amazon Kinesis||Stream State|
|Amazon Machine Learning||BatchPrediction, DataSource, Evaluation , MLModel|
|Amazon Route 53||Domains, Healthchecks , HostedZone|
|AWS Elastic Load Balancing||Loadbalancer, TargetGroups|
|AWS Identity and Access Management||Profile Name|
|AWS SQS||Queue Name|
|Apache||Apache Host and Port|
|Azure||Tenant Name, Status, Tags, Subscription ID and Name, Availability Zone in common with AWS tag after contacting Datadog support|
|BTRFS||Usage & Replication Type|
|Consul||Previous and Current Consul Leaders and Followers, Consul Datacenter, Service Name, Service ID|
|CouchDB||Database Name, Instance Name|
|CouchBase||CouchBase Tags, Instance Name|
|Docker||Docker Container and Image Name, Container Command, Container Labels|
|Dyn||Zone, Record Type|
|Elasticsearch||Cluster Name, Host Name, Port Number|
|Etcd||State Leader or Follower|
|Fluentd||Host Name, Port Number|
|Google App Engine||Project Name, Version ID, Task Queue|
|Google Cloud Platform||Zone, Instance Type and ID, Automatic Restart, Project Name and ID, Name, Availability Zone in common with AWS tag after contacting Datadog support|
|Go Expvar||Expvar Path|
|Gunicorn||State Idle or Working, App Name|
|HAProxy||Service Name, Availability, Backend Host, Status, Type|
|HTTP Check||URL, Instance|
|Jenkins||Job Name, Build Number, Branch, and Results|
|Kubernetes||Minion Name, Namespace, Replication Controller, Labels, Container Alias|
|Memcached||Host, Port, Request, Cache Hit or Miss|
|Mesos||Role, URL, PID, Slave or Master Role, Node, Cluster,|
|OpenStack||Network ID, Network Name, Hypervisor Name, ID, and Type, Tenant ID, Availability Zone|
|PHP FPM||Pool Name|
|Pivotal||Current State, Owner, Labels, Requester, Story Type|
|RabbitMQ||Node, Queue Name, Vhost, Policy, Host|
|Redis||Host, Port, Slave or Master|
|SNMP||Device IP Address|
|Supervisord||Server Name, Process Name|
|TeamCity||Tags, Code Deployments, Build Number|
|TokuMX||Role Primary or Secondary, Replset, Replstate, Db, Coll, Shard|
|VSphere||Host, Datacenter, Server, Instance|
|Win32 Events||Event ID|
|Windows Services||Service Name|
The Datadog integrations are all configured via the yaml configuration files located in the conf.d directory in your agent install. For more about where to look for your configuration files, refer to this article. You can define tags in the configuration file for the overall agent as well as for each integration, though the datadog.conf file is a more traditional ini file. In yaml files, there is a tag dictionary with a list of tags you want assigned at that level. Any tag you assign to the agent will apply to every integration on that agent’s host.
Dictionaries with lists of values have two different yet functionally equivalent forms:
tags: firsttag, secondtag, thirdtag
tags: - firsttag - secondtag - thirdtag
You will see both forms in the yaml configuration files, but for the datadog.conf ini file only the first form is valid.
Each tag can be anything you like but you will have the best success with tagging if your tags are key:value pairs. Keys could represent the role, or function, or region, or application and the value is the instance of that role, function, region, or application. Here are some examples of good tags:
region:east region:nw application:database database:primary role:sobotka
The reason why you should use key value pairs instead of simply values will become apparent when you start using the tags to filter and group metrics and machines. That said, you are not required to use key value pairs and simple values are valid.
You can also assign tags to hosts, but not to integrations in the UI. To assign tags in the UI, start by going to the Infrastructure List page. Click on any host and then click the Update Host Tags button. In the host overlay that appears, click Edit Tags and make the changes you wish.
You can also assign tags to hosts, but not to integrations using the API. The endpoints you want to work with are /tags/hosts and depending on whether you PUT, POST, or DELETE you will update, add, or delete tags for the chosen host. For more details on using the Tags endpoints in the API, review this document
After you have assigned tags at the host and integration level, you can start using them to filter and group in interesting ways. There are several places you can use tags:
The Events List will show you all the events that have occured in your environment over the time period specified. This can be overwhelming so you can use tags to filter down the list based on the tags you have assigned. You can enter any text you want in the search box above the Event List and a full text search will be performed. You can also enter
tags: followed by a tag to see all the events that come from a host or integration with that tag. The example in the image is the tag role:cassandra. So the search text is
You can use tags to narrow down the metrics to display on a dashboard graph, or to create groups of metrics to display. To narrow down the metrics to display, enter the tag in the
from: textbox. You will now be looking at a chosen metric over all the hosts that have that particular tag assigned.
To group using tags, enter the key part of the tag in the
avg by: textbox.
For instance, if you have a time series graph showing a metric tagged by the reporting hosts’ roles —
role:loadbalancer— enter role in the avg_by textbox. This causes the graph to show just one line for each tag value —
loadbalancer. Each line represents the average metric value across all hosts that share that role.
You can also use tags to overlay events on the dashboard. This works in exactly the same way as in the Events List. Simply enter
tags: followed by the tag and you will see the corresponding events overlaid as vertical bars on each graph.
To filter the list of hosts in the Infrastructure list, enter a tag in the filter textbox at the top of the page. You can also group the hosts by entering the key portion of the tag in the group by textbox. So if you enter role in the group box, you will see each role as a group heading followed by the hosts with that tag.
When creating a monitor:
Use tags in the
from: textbox to limit the monitor scope to only metrics that have those tags.
Use tags in the
excluding: textbox to remove the corresponding metrics of the monitor scope.
Use tags in the
avg by textbox transform your monitor into a multi-alert monitor on each value of this tags. Tags on these events are related to the
avg by: value. In order to have host-related tags (such as AWS integration tags), use
avg by: host