Overview
Use the information below for troubleshooting Datadog Network Device Monitoring. If you need additional help, contact Datadog support.
Device not visible in Datadog
The following assumes you are running Datadog Agent v7.59.0+.
If your device is not visible on the Devices page:
Run the datadog-agent status command and look for the snmp section, which contains your device’s monitoring IP. After you start the Agent, it may take up to one minute for NDM to discover individually configured devices. If your Agent is set to scan a large number of devices, it may take longer.
The output should look similar to the following:
snmp
----
Instance ID: snmp:default:1.2.3.4.1:9a2df638d3ba38d6 [ERROR]
Configuration Source: file:/etc/datadog-agent/conf.d/snmp.d/conf.yaml
Total Runs: 1
Metric Samples: Last Run: 6, Total: 6
Events: Last Run: 0, Total: 0
Network Devices Metadata: Last Run: 1, Total: 1
Service Checks: Last Run: 1, Total: 1
Average Execution Time : 0s
Last Execution Date : 2024-11-13 13:12:09 PST / 2024-11-13 21:12:09 UTC (1731532329000)
Last Successful Execution Date : Never
Error: <ERROR MESSAGE>
No traceback
If your device is not listed, and you are using Autodiscovery, it likely means the Agent could not connect to your device. Try running an snmp walk on the device’s admin IP.
Note: Provide your credentials directly in the CLI. If credentials aren’t provided, the Agent attempts to locate them in your running Agent configuration files.
Linux:
SNMP v2:
sudo -u dd-agent datadog-agent snmp walk <IP Address> -C <COMMUNITY_STRING>
SNMP v3:
sudo -u dd-agent datadog-agent snmp walk <IP Address> -A <AUTH_KEY> -a <AUTH_PROTOCOL> -X <PRIV_KEY> -x <PRIV_PROTOCOL>
Windows:
agent snmp walk <IP Address>[:Port]
Example:
agent.exe snmp walk 10.143.50.30 1.3.6
Refer to your vendor specific documentation for additional information on running these commands.
Troubleshooting SNMP Errors
If either the SNMP status or Agent walk shows an error, it could indicate one of the following issues:
Permission denied
If you see a permission denied error while port binding in agent logs, the port number you’ve indicated may require elevated permissions. To bind to a port number under 1024, see Using the default SNMP Trap port 162.
Error:
Error: check device reachable: failed: error reading from socket: read udp 127.0.0.1:46068->1.2.3.4:161
Solution:
Log into your device and ensure that SNMP is enabled and exposed on port 161.
Verify that your collector firewall allows egress.
Optionally, For Linux only:
Run iptables -L OUTPUT and ensure there is no deny rule:
vagrant@agent-dev-ubuntu-22:~$ sudo iptables -L OUTPUT
Chain OUTPUT (policy ACCEPT)
target prot opt source destination
DROP all -- anywhere 10.4.5.6
Ensure your community string matches.
Incorrect SNMPv2 credentials
Error:
Error: an authentication method needs to be provided
Solution:
If using SNMPv2, ensure that a community string is set.
Incorrect SNMPv3 privacy protocol
Error:
Error: check device reachable: failed: decryption error; failed to autodetect profile: failed to fetch sysobjectid: cannot get sysobjectid: decryption error; failed to fetch values: failed to fetch scalar oids with batching: failed to fetch scalar oids: fetch scalar: error getting oids `[1.3.6.1.2.1.1.1.0 1.3.6.1.2.1.1.2.0 1.3.6.1.2.1.1.3.0 1.3.6.1.2.1.1.5.0]`: decryption error
OR
Error: check device reachable: failed: wrong digest; failed to autodetect profile: failed to fetch sysobjectid: cannot get sysobjectid: wrong digest; failed to fetch values: failed to fetch scalar oids with batching: failed to fetch scalar oids: fetch scalar: error getting oids `[1.3.6.1.2.1.1.1.0 1.3.6.1.2.1.1.2.0 1.3.6.1.2.1.1.3.0 1.3.6.1.2.1.1.5.0]`: wrong digest
Solution:
Verify that the following SNMPv3 configuration parameters are correct:
- user
- authKey
- authProtocol
- privKey
- privProtocol
Traps not being received for devices
Check the Datadog agent.log file to ensure that you can bind to the traps port. The following error indicates that you are unable to bind to the traps port:
Failed to start snmp-traps server: error happened when listening for SNMP Traps: listen udp 0.0.0.0:162: bind: permission denied
Solution:
Add a net bind capability to the Agent binary, which allows the Agent to bind to reserved ports:
sudo setcap 'cap_net_bind_service=+ep' /opt/datadog-agent/bin/agent/agent
Navigate to the troubleshooting dashboard in NDM:
Scroll down to the Traps widget and observe the Traps incorrectly formatted graph. If this is non-zero it likely means that the authentication on the NDM collector and the device do not match.
Solution:
Verify that the following configurations in the datadog.yaml file align with the trap settings on the devices from which traps are missing:
## @param community_strings - list of strings - required
## A list of known SNMP community strings that devices can use to send traps to the Agent.
## Traps with an unknown community string are ignored.
## Enclose the community string with single quote like below (to avoid special characters being interpreted).
## Must be non-empty.
#
# community_strings:
# - '<COMMUNITY_1>'
# - '<COMMUNITY_2>'
## @param users - list of custom objects - optional
## List of SNMPv3 users that can be used to listen for traps.
## Each user can contain:
## * user - string - The username used by devices when sending Traps to the Agent.
## * authKey - string - (Optional) The passphrase to use with the given user and authProtocol
## * authProtocol - string - (Optional) The authentication protocol to use when listening for traps from this user.
## Available options are: MD5, SHA, SHA224, SHA256, SHA384, SHA512.
## Defaults to MD5 when authKey is set.
## * privKey - string - (Optional) The passphrase to use with the given user privacy protocol.
## * privProtocol - string - (Optional) The privacy protocol to use when listening for traps from this user.
## Available options are: DES, AES (128 bits), AES192, AES192C, AES256, AES256C.
## Defaults to DES when privKey is set.
#
# users:
# - user: <USERNAME>
# authKey: <AUTHENTICATION_KEY>
# authProtocol: <AUTHENTICATION_PROTOCOL>
# privKey: <PRIVACY_KEY>
# privProtocol: <PRIVACY_PROTOCOL>
Further Reading
Additional helpful documentation, links, and articles:
