Power DNS Recursor

Supported OS Linux Windows Mac OS

Integration version4.0.0

Overview

Track the performance of your PowerDNS Recursor and monitor strange or worrisome traffic. This Agent check collects a variety of metrics from your recursors, including those for:

  • Query answer times-see how many responses take less than 1ms, 10ms, 100ms, 1s, or greater than 1s.
  • Query timeouts.
  • Cache hits and misses.
  • Answer rates by type: SRVFAIL, NXDOMAIN, NOERROR.
  • Ignored and dropped packets.

And many more.

Setup

Installation

The PowerDNS Recursor check is included in the Datadog Agent package, so you don’t need to install anything else on your recursors.

Configuration

Prepare PowerDNS

This check collects performance statistics using PowerDNS Recursor’s statistics API. Versions of pdns_recursor before 4.1 do not enable the stats API by default. If you’re running an older version, enable it by adding the following to your recursor config file, for example /etc/powerdns/recursor.conf:

webserver=yes
api-key=changeme             # only available since v4.0
webserver-readonly=yes       # default no
#webserver-port=8081         # default 8082
#webserver-address=0.0.0.0   # default 127.0.0.1

If you’re running pdns_recursor 3.x, prepend experimental- to these option names, for example: experimental-webserver=yes.

If you’re running pdns_recursor >= 4.1, just set api-key.

Restart the recursor to enable the statistics API.

Host

To configure this check for an Agent running on a host:

  1. Edit the powerdns_recursor.d/conf.yaml file, in the conf.d/ folder at the root of your Agent’s configuration directory. See the sample powerdns_recursor.d/conf.yaml for all available configuration options:

    init_config:
    
    instances:
      ## @param host - string - required
      ## Host running the recursor.
      #
      - host: 127.0.0.1
    
        ## @param port - integer - required
        ## Recursor web server port.
        #
        port: 8082
    
        ## @param api_key - string - required
        ## Recursor web server api key.
        #
        api_key: "<POWERDNS_API_KEY>"
    
        ## @param version - integer - required - default: 3
        ## Version 3 or 4 of PowerDNS Recursor to connect to.
        ## The PowerDNS Recursor in v4 has a production ready web server that allows for
        ## statistics gathering. In version 3.x the server was marked as experimental.
        ##
        ## As the server was marked as experimental in version 3 many of the metrics have
        ## changed names and the API structure (paths) have also changed. With these changes
        ## there has been a need to separate the two concerns. The check now has a key value
        ## version: which if set to version 4 queries with the correct API path on the
        ## non-experimental web server.
        ##
        ## https://doc.powerdns.com/md/httpapi/api_spec/#url-apiv1serversserver95idstatistics
        #
        version: 3
    
  2. Restart the Agent.

Log collection
  1. Collecting logs is disabled by default in the Datadog Agent, you need to enable it in datadog.yaml:

    logs_enabled: true
    
  2. Add the dd-agent user to the systemd-journal group by running:

    usermod -a -G systemd-journal dd-agent
    
  3. Add this configuration block to your powerdns_recursor.d/conf.yaml file to start collecting your PowerDNS Recursor Logs:

    logs:
      - type: journald
        source: powerdns
    

    See the sample powerdns_recursor.d/conf.yaml for all available configuration options.

  4. Restart the Agent.

Containerized

For containerized environments, see the Autodiscovery Integration Templates for guidance on applying the parameters below.

ParameterValue
<INTEGRATION_NAME>powerdns_recursor
<INIT_CONFIG>blank or {}
<INSTANCE_CONFIG>{"host":"%%host%%", "port":8082, "api_key":"<POWERDNS_API_KEY>", "version": 3}
Log collection

Collecting logs is disabled by default in the Datadog Agent. To enable it, see Kubernetes Log Collection.

ParameterValue
<LOG_CONFIG>{"source": "powerdns"}

Validation

Run the Agent’s status subcommand and look for powerdns_recursor under the Checks section.

Data Collected

Metrics

powerdns.recursor.all_outqueries
(gauge)
The number of outgoing udp queries per second
Shown as query
powerdns.recursor.answers0_1
(gauge)
Number of queries per second answered within 1 millisecond
Shown as query
powerdns.recursor.answers100_1000
(gauge)
Number of queries per second answered within 1 second
Shown as query
powerdns.recursor.answers10_100
(gauge)
Number of queries per second answered within 100 milliseconds
Shown as query
powerdns.recursor.answers1_10
(gauge)
Number of queries per second answered within 10 milliseconds
Shown as query
powerdns.recursor.answers_slow
(gauge)
Number of queries per second NOT answered within 1 second
Shown as query
powerdns.recursor.auth4_answers0_1
(gauge)
Number of queries per second answered by auth4s within 1 millisecond; Available since pdns_recursor v4.x
Shown as query
powerdns.recursor.auth4_answers100_1000
(gauge)
Number of queries per second answered by auth4s within 1 second; Available since pdns_recursor v4.x
Shown as query
powerdns.recursor.auth4_answers10_100
(gauge)
Number of queries per second answered by auth4s within 100 milliseconds; Available since pdns_recursor v4.x
Shown as query
powerdns.recursor.auth4_answers1_10
(gauge)
Number of queries per second answered by auth4s within 10 milliseconds; Available since pdns_recursor v4.x
Shown as query
powerdns.recursor.auth4_answers_slow
(gauge)
Number of queries per second NOT answered by auth4s within 1 second; Available since pdns_recursor v4.x
Shown as query
powerdns.recursor.auth6_answers0_1
(gauge)
Number of queries per second answered by auth6s within 1 millisecond; Available since pdns_recursor v4.x
Shown as query
powerdns.recursor.auth6_answers100_1000
(gauge)
Number of queries per second answered by auth6s within 1 second; Available since pdns_recursor v4.x
Shown as query
powerdns.recursor.auth6_answers10_100
(gauge)
Number of queries per second answered by auth6s within 100 milliseconds; Available since pdns_recursor v4.x
Shown as query
powerdns.recursor.auth6_answers1_10
(gauge)
Number of queries per second answered by auth6s within 10 milliseconds; Available since pdns_recursor v4.x
Shown as query
powerdns.recursor.auth6_answers_slow
(gauge)
Number of queries per second NOT answered by auth6s within 1 second; Available since pdns_recursor v4.x
Shown as query
powerdns.recursor.cache_entries
(gauge)
The number of entries in the cache
Shown as entry
powerdns.recursor.cache_hits
(gauge)
The number of cache hits per second
Shown as hit
powerdns.recursor.cache_misses
(gauge)
The number of cache misses per second
Shown as miss
powerdns.recursor.case_mismatches
(gauge)
The number of mismatches in character case per second
Shown as error
powerdns.recursor.chain_resends
(gauge)
The number of queries per second chained to existing outstanding query
Shown as query
powerdns.recursor.client_parse_errors
(gauge)
The number of unparsable packets per second
Shown as error
powerdns.recursor.concurrent_queries
(gauge)
The number of MThreads currently running
Shown as query
powerdns.recursor.dlg_only_drops
(gauge)
The number of records dropped per second because of 'delegation only' setting; Available since pdns_recursor v4.x
Shown as record
powerdns.recursor.dnssec_queries
(gauge)
The number of queries received per second with the DO bit set; Available since pdns_recursor v4.x
Shown as record
powerdns.recursor.dnssec_result_bogus
(gauge)
The number of DNSSEC validations per second that had the Bogus state; Available since pdns_recursor v4.x
powerdns.recursor.dnssec_result_indeterminate
(gauge)
The number of DNSSEC validations per second that had the Indeterminate state; Available since pdns_recursor v4.x
powerdns.recursor.dnssec_result_insecure
(gauge)
The number of DNSSEC validations per second that had the Insecure state; Available since pdns_recursor v4.x
powerdns.recursor.dnssec_result_nta
(gauge)
The number of DNSSEC validations per second that had the NTA (negative trust anchor) state; Available since pdns_recursor v4.x
powerdns.recursor.dnssec_result_secure
(gauge)
The number of DNSSEC validations per second that had the Secure state; Available since pdns_recursor v4.x
powerdns.recursor.dnssec_validations
(gauge)
The number of DNSSEC validations performed per second; Available since pdns_recursor v4.x
powerdns.recursor.dont_outqueries
(gauge)
The number of outgoing queries dropped per second because of 'dont query' setting
Shown as query
powerdns.recursor.edns_ping_matches
(gauge)
The number of servers per second that sent a valid EDNS PING response; Available since pdns_recursor v4.x
Shown as host
powerdns.recursor.edns_ping_mismatches
(gauge)
The number of servers per second that sent an invalid EDNS PING response; Available since pdns_recursor v4.x
Shown as host
powerdns.recursor.failed_host_entries
(gauge)
The number of servers that failed to resolve
Shown as host
powerdns.recursor.ignored_packets
(gauge)
The number of non-query packets received per second on server sockets that should only get queries; Available since pdns_recursor v4.x
Shown as packet
powerdns.recursor.ipv6_outqueries
(gauge)
The number of outgoing queries per second over IPv6
powerdns.recursor.ipv6_questions
(gauge)
The number of end-user-initiated IPv6 UDP queries with the RD bit set
Shown as query
powerdns.recursor.max_mthread_stack
(gauge)
The maximum amount of thread stack ever used
powerdns.recursor.negcache_entries
(gauge)
The number of entries in the negative answer cache
Shown as entry
powerdns.recursor.no_error_packets
(gauge)
The number of erroneous packets received per second; Available since pdns_recursor v4.x
Shown as packet
powerdns.recursor.noedns_outqueries
(gauge)
The number of queries per second sent without EDNS; Available since pdns_recursor v4.x
Shown as query
powerdns.recursor.noerror_answers
(gauge)
The number of NOERROR answers per second
Shown as operation
powerdns.recursor.noping_outqueries
(gauge)
The number of queries per second sent without EDNS PING; Available since pdns_recursor v4.x
Shown as query
powerdns.recursor.nsset_invalidations
(gauge)
The number of times per second an nsset was dropped because it no longer worked; Available since pdns_recursor v4.x
powerdns.recursor.nsspeeds_entries
(gauge)
The number of entries in the NS speeds map; Available since pdns_recursor v4.x
Shown as entry
powerdns.recursor.nxdomain_answers
(gauge)
The number of NXDOMAIN answers per second
Shown as response
powerdns.recursor.outgoing4_timeouts
(gauge)
The number of timeouts per second for outgoing UDP IPv4 queries; Available since pdns_recursor v4.x
Shown as timeout
powerdns.recursor.outgoing6_timeouts
(gauge)
The number of timeouts per second for outgoing UDP IPv6 queries; Available since pdns_recursor v4.x
Shown as timeout
powerdns.recursor.outgoing_timeouts
(gauge)
The number of outgoing UDP query timeouts per second
Shown as timeout
powerdns.recursor.over_capacity_drops
(gauge)
The number of questions per second dropped due to having reached concurrent query limit
Shown as query
powerdns.recursor.packetcache_entries
(gauge)
The number of entries in the packet cache
Shown as entry
powerdns.recursor.packetcache_hits
(gauge)
The number of packet cache hits per second
Shown as hit
powerdns.recursor.packetcache_misses
(gauge)
The number of packet cache misses per second
Shown as miss
powerdns.recursor.policy_drops
(gauge)
The number of packets dropped per second because of Lua policy decision
Shown as packet
powerdns.recursor.policy_result_custom
(gauge)
The number of packets per second that were sent a custom answer by the RPZ/filter engine; Available since pdns_recursor v4.x
Shown as packet
powerdns.recursor.policy_result_drop
(gauge)
The number of packets per second dropped by the RPZ/filter engine; Available since pdns_recursor v4.x
Shown as packet
powerdns.recursor.policy_result_noaction
(gauge)
The number of packets per second that were not actioned upon by the RPZ/filter engine; Available since pdns_recursor v4.x
Shown as packet
powerdns.recursor.policy_result_nodata
(gauge)
The number of packets per second that were replied to with NODATA by the RPZ/filter engine; Available since pdns_recursor v4.x
Shown as packet
powerdns.recursor.policy_result_nxdomain
(gauge)
The number of packets per second that were replied to with NXDOMAIN by the RPZ/filter engine; Available since pdns_recursor v4.x
Shown as packet
powerdns.recursor.policy_result_truncate
(gauge)
The number of packets per second that were forced to TCP by the RPZ/filter engine; Available since pdns_recursor v4.x
Shown as packet
powerdns.recursor.qa_latency
(gauge)
The average latency in microseconds exponentially weighted over past 'latency-statistic-size' packets
Shown as microsecond
powerdns.recursor.questions
(gauge)
The number of user initiated udp queries per second
Shown as operation
powerdns.recursor.real_memory_usage
(gauge)
The amount of memory consumed by PowerDNS in bytes; Available since pdns_recursor v4.x
Shown as byte
powerdns.recursor.resource_limits
(gauge)
The number of queries per second that could not be performed due to resource limits; Available since pdns_recursor v4.x
Shown as query
powerdns.recursor.server_parse_errors
(gauge)
The number of server replied packets per second that could not be parsed
Shown as error
powerdns.recursor.servfail_answers
(gauge)
The number of SERVFAIL answers per second
Shown as response
powerdns.recursor.spoof_prevents
(gauge)
The number of times per second PowerDNS considers itself spoofed and drops data
powerdns.recursor.sys_msec
(gauge)
The number of CPU milliseconds spent in 'system' mode
Shown as millisecond
powerdns.recursor.tcp_client_overflow
(gauge)
The number of outgoing queries dropped per second because of 'dont query' setting
Shown as query
powerdns.recursor.tcp_clients
(gauge)
The number of active TCP/IP clients per second
powerdns.recursor.tcp_outqueries
(gauge)
The number of outgoing TCP queries per second
Shown as operation
powerdns.recursor.tcp_questions
(gauge)
The number of incoming tcp queries per second
Shown as operation
powerdns.recursor.throttle_entries
(gauge)
The number of entries in the throttle map
Shown as entry
powerdns.recursor.throttled_out
(gauge)
The number of throttled outgoing UDP queries per second
Shown as operation
powerdns.recursor.too_old_drops
(gauge)
The number of questions per second that were dropped because they were too old; Available since pdns_recursor v4.x
Shown as query
powerdns.recursor.udp_in_errors
(gauge)
The number of packets per second that were received faster than the OS could process them; Available since pdns_recursor v4.x
Shown as packet
powerdns.recursor.udp_noport_errors
(gauge)
The number of UDP packets per second where an ICMP response was received saying the remote port was not listening; Available since pdns_recursor v4.x
Shown as packet
powerdns.recursor.udp_recvbuf_errors
(gauge)
The number of errors per second caused in the UDP receive buffer; Available since pdns_recursor v4.x
Shown as error
powerdns.recursor.udp_sndbuf_errors
(gauge)
The number of errors per second caused in the UDP send buffer; Available since pdns_recursor v4.x
Shown as error
powerdns.recursor.unauthorized_tcp
(gauge)
The number of TCP questions denied per second because of allow-from restrictions
Shown as operation
powerdns.recursor.unauthorized_udp
(gauge)
The number of UDP questions denied per second because of allow-from restrictions
Shown as operation
powerdns.recursor.unexpected_packets
(gauge)
The number of unexpected answers per second from remote servers
Shown as operation
powerdns.recursor.unreachables
(gauge)
The number times per second nameservers were unreachable
powerdns.recursor.uptime
(gauge)
The number of seconds PowerDNS has been running
Shown as second
powerdns.recursor.user_msec
(gauge)
The number of CPU milliseconds spent in 'user' mode
Shown as millisecond

Events

The PowerDNS Recursor check does not include any events.

Service Checks

powerdns.recursor.can_connect
Returns CRITICAL if the Agent check is unable to connect to the monitored Gearman instance. Returns OK otherwise.
Statuses: ok, critical

Troubleshooting

Need help? Contact Datadog support.