Storm

Supported OS Linux Windows Mac OS

Integration version1.0.1

Overview

Get metrics from Storm service in real time to:

  • Visualize and monitor Storm cluster and topology metrics.
  • Be notified about Storm failovers and events.

Setup

The Storm check is not included in the Datadog Agent package, so you need to install it.

Installation

For Agent v7.21+ / v6.21+, follow the instructions below to install the Storm check on your host. See Use Community Integrations to install with the Docker Agent or earlier versions of the Agent.

  1. Run the following command to install the Agent integration:

    datadog-agent integration install -t datadog-storm==<INTEGRATION_VERSION>

  2. Configure your integration similar to core integrations.

Configuration

  1. Edit the storm.d/conf.yaml file in the conf.d/ folder at the root of your Agent’s configuration directory to start collecting your Storm metrics. See the sample storm.d/conf.yaml for all available configuration options.

  2. Restart the Agent.

Validation

Run the Agent’s status subcommand and look for storm under the Checks section.

Data Collected

Metrics

storm.bolt.last_60.acked
(gauge)		Number of Acked Tuples
Shown as sample
storm.bolt.last_60.capacity
(gauge)		Bolt Capacity
Shown as fraction
storm.bolt.last_60.emitted
(gauge)		Number of Emitted Tuples
Shown as sample
storm.bolt.last_60.errorLapsedSecs
(gauge)		Number of Seconds Since Last Error
Shown as second
storm.bolt.last_60.executed
(gauge)		Number of Tuples Executed
Shown as sample
storm.bolt.last_60.executeLatency
(gauge)		Bolt Execute Latency
Shown as millisecond
storm.bolt.last_60.executors
(gauge)		Number of Bolt Executors
Shown as thread
storm.bolt.last_60.failed
(gauge)		Number of Failed Tuples
Shown as sample
storm.bolt.last_60.processLatency
(gauge)		Bolt Process Latency
Shown as millisecond
storm.bolt.last_60.requestedCpu
(gauge)		Bolt Requested CPU
Shown as percent
storm.bolt.last_60.requestedMemOffHeap
(gauge)		Bolt Requested Memory Off Heap
Shown as mebibyte
storm.bolt.last_60.requestedMemOnHeap
(gauge)		Bolt Requested Memory On Heap
Shown as mebibyte
storm.bolt.last_60.tasks
(gauge)		Bolt Tasks
Shown as task
storm.bolt.last_60.transferred
(gauge)		Number of Transferred Tuples
Shown as sample
storm.cluster.availCpu
(gauge)		Available Storm Cluster CPU
Shown as core
storm.cluster.availMem
(gauge)		Available Storm Cluster Memory
Shown as mebibyte
storm.cluster.cpuAssignedPercentUtil
(gauge)		Storm Cluster CPU Assigned Percent
Shown as percent
storm.cluster.executorsTotal
(gauge)		Total Storm Cluster Executors
Shown as thread
storm.cluster.memAssignedPercentUtil
(gauge)		Storm Cluster Memory Assigned Percent
Shown as percent
storm.cluster.slotsFree
(gauge)		Total Cluster Slots Available
Shown as process
storm.cluster.slotsTotal
(gauge)		Total Cluster Slots
Shown as process
storm.cluster.slotsUsed
(gauge)		Total Storm Cluster Slots Used
Shown as process
storm.cluster.supervisors
(gauge)		Total Storm Cluster Supervisors
Shown as worker
storm.cluster.tasksTotal
(gauge)		Total Storm Cluster Tasks
Shown as task
storm.cluster.topologies
(gauge)		Number of Storm Topologies
Shown as service
storm.cluster.totalCpu
(gauge)		Total Storm Cluster CPU
Shown as core
storm.cluster.totalMem
(gauge)		Total Storm Cluster Memory
Shown as mebibyte
storm.nimbus.numDead
(gauge)		Number of Dead Nimbus Nodes
Shown as node
storm.nimbus.numFollowers
(gauge)		Number of Follower Nimbus Nodes
Shown as node
storm.nimbus.numLeaders
(gauge)		Number of Leader Nimbus Nodes
Shown as node
storm.nimbus.numOffline
(gauge)		Number of Offline Nimbus Nodes
Shown as node
storm.nimbus.upTimeSeconds
(gauge)		Nimbus Uptime Seconds
Shown as second
storm.spout.last_60.acked
(gauge)		Number of Acked Tuples
Shown as sample
storm.spout.last_60.completeLatency
(gauge)		Spout Complete Latency
Shown as millisecond
storm.spout.last_60.emitted
(gauge)		Number of Emitted Tuples
Shown as sample
storm.spout.last_60.errorLapsedSecs
(gauge)		Number of Seconds Since Last Error
Shown as second
storm.spout.last_60.executors
(gauge)		Number of Spout Executors
Shown as thread
storm.spout.last_60.failed
(gauge)		Number of Failed Tuples
Shown as sample
storm.spout.last_60.requestedCpu
(gauge)		Spout Requested CPU
Shown as percent
storm.spout.last_60.requestedMemOffHeap
(gauge)		Spout Requested Memory Off Heap
Shown as mebibyte
storm.spout.last_60.requestedMemOnHeap
(gauge)		Spout Requested Memory On Heap
Shown as mebibyte
storm.spout.last_60.tasks
(gauge)		Spout Tasks
Shown as task
storm.spout.last_60.transferred
(gauge)		Number of Transferred Tuples
Shown as sample
storm.supervisor.slotsTotal
(gauge)		Total Supervisor Slots
Shown as process
storm.supervisor.slotsUsed
(gauge)		Used Supervisor Slots
Shown as process
storm.supervisor.totalCpu
(gauge)		Total Supervisor CPU
Shown as core
storm.supervisor.totalMem
(gauge)		Total Supervisor Memory
Shown as mebibyte
storm.supervisor.uptimeSeconds
(gauge)		Supervisor Uptime
Shown as second
storm.supervisor.usedCpu
(gauge)		Used Supervisor CPU
Shown as core
storm.supervisor.usedMem
(gauge)		Used Supervisor Memory
Shown as mebibyte
storm.topologyStats.last_60.acked
(gauge)		All Time Acked Tuples
Shown as sample
storm.topologyStats.last_60.assignedCpu
(gauge)		Assigned CPU Percentage
Shown as percent
storm.topologyStats.last_60.assignedMemOffHeap
(gauge)		Off Heap Memory Assigned
Shown as mebibyte
storm.topologyStats.last_60.assignedMemOnHeap
(gauge)		On Heap Memory Assigned
Shown as mebibyte
storm.topologyStats.last_60.assignedTotalMem
(gauge)		Total Memory Assigned
Shown as mebibyte
storm.topologyStats.last_60.completeLatency
(gauge)		All Time Complete Latence
Shown as millisecond
storm.topologyStats.last_60.debug
(gauge)		Boolean indicating if debug mode is enabled.
Shown as sample
storm.topologyStats.last_60.emitted
(gauge)		All Time Emitted Tuples
Shown as sample
storm.topologyStats.last_60.executorsTotal
(gauge)		Total Storm Topology Executors
Shown as thread
storm.topologyStats.last_60.failed
(gauge)		All Time Failed Tuples
Shown as sample
storm.topologyStats.last_60.msgTimeout
(gauge)		Spout Tuple Timeout in Seconds
Shown as second
storm.topologyStats.last_60.numBolts
(gauge)		Total Number of Bolts
Shown as task
storm.topologyStats.last_60.numSpouts
(gauge)		Total Number of Spouts
Shown as task
storm.topologyStats.last_60.replicationCount
(gauge)		Number of Replications
Shown as occurrence
storm.topologyStats.last_60.requestedCpu
(gauge)		Requested Topology CPU resources
Shown as percent
storm.topologyStats.last_60.requestedMemOffHeap
(gauge)		Requested Topology Off Heap Memory resources
Shown as mebibyte
storm.topologyStats.last_60.requestedMemOnHeap
(gauge)		Requested Topology On Heap Memory Resources
Shown as mebibyte
storm.topologyStats.last_60.samplingPct
(gauge)		Metric Sampling Percentage by Storm
Shown as percent
storm.topologyStats.last_60.tasksTotal
(gauge)		Total Number of Tasks
Shown as task
storm.topologyStats.last_60.transferred
(gauge)		All Time Transferred Tuples
Shown as sample
storm.topologyStats.last_60.uptimeSeconds
(gauge)		Total Topology Uptime
Shown as second
storm.topologyStats.last_60.workersTotal
(gauge)		Total Number of Workers
Shown as worker
storm.topologyStats.metrics.bolts.last_60..acked
(gauge)		Number of Tuples Acked by Spout & Stream
Shown as sample
storm.topologyStats.metrics.bolts.last_60.complete_ms_avg
(gauge)		Complete Tuple Latency by Spout & Stream
Shown as millisecond
storm.topologyStats.metrics.bolts.last_60.emitted
(gauge)		Number of Tuples Emitted by Spout & Stream
Shown as sample
storm.topologyStats.metrics.bolts.last_60.executed
(gauge)		Number of Tuples Executed by Spout & Stream
Shown as sample
storm.topologyStats.metrics.bolts.last_60.executed_ms_avg
(gauge)		Execute Tuple Latency by Spout & Stream
Shown as millisecond
storm.topologyStats.metrics.bolts.last_60.failed
(gauge)		Number of Tuples Failed by Spout & Stream
Shown as sample
storm.topologyStats.metrics.bolts.last_60.process_ms_avg
(gauge)		Process Tuple Latency by Spout & Stream
Shown as millisecond
storm.topologyStats.metrics.bolts.last_60.transferred
(gauge)		Number of Tuples Transferred by Spout & Stream
Shown as sample
storm.topologyStats.metrics.spouts.last_60.acked
(gauge)		Number of Tuples Acked by Spout & Stream
Shown as sample
storm.topologyStats.metrics.spouts.last_60.complete_ms_avg
(gauge)		Complete Tuple Latency by Spout & Stream
Shown as millisecond
storm.topologyStats.metrics.spouts.last_60.emitted
(gauge)		Number of Tuples Emitted by Spout & Stream
Shown as sample
storm.topologyStats.metrics.spouts.last_60.executed
(gauge)		Number of Tuples Executed by Spout & Stream
Shown as sample
storm.topologyStats.metrics.spouts.last_60.executed_ms_avg
(gauge)		Execute Tuple Latency by Spout & Stream
Shown as millisecond
storm.topologyStats.metrics.spouts.last_60.failed
(gauge)		Number of Tuples Failed by Spout & Stream
Shown as sample
storm.topologyStats.metrics.spouts.last_60.process_ms_avg
(gauge)		Process Tuple Latency by Spout & Stream
Shown as millisecond
storm.topologyStats.metrics.spouts.last_60.transferred
(gauge)		Number of Tuples Transferred by Spout & Stream
Shown as sample
storm.worker.last_60.assignedCpu
(gauge)		Assigned Worker CPU Percentage
Shown as percent
storm.worker.last_60.assignedMemOffHeap
(gauge)		Off Heap Memory Assigned for a Worker
Shown as mebibyte
storm.worker.last_60.assignedMemOnHeap
(gauge)		On Heap Memory Assigned for a Worker
Shown as mebibyte
storm.worker.last_60.componentNumTasks
(gauge)		Total Number of Component Tasks for a Worker
Shown as task
storm.worker.last_60.executorsTotal
(gauge)		Total Number of Executors for a Worker
Shown as thread
storm.worker.last_60.uptimeSeconds
(gauge)		Worker Uptime
Shown as second

Events

The Storm check does not include any events.

Service Checks

Troubleshooting

Need help? Contact Datadog support.