Sidekiq
New announcements from Dash: Incident Management, Continuous Profiler, and more! New announcements from Dash!

Sidekiq

Agent Check Agent Check

Supported OS: Linux Mac OS Windows

Overview

This integration monitors Sidekiq through Dogstatsd. It collects metrics through Datadog’s Dogstatsd Ruby client.

Note Only Sidekiq Pro (>= 3.6) or Enterprise (>= 1.1.0) users can collect metrics.

Setup

Installation

The Sidekiq integration is packaged with the Datadog Agent. No additional installation is needed on your server.

Configuration

  1. Install the dogstatsd-ruby gem:

    gem install dogstatsd-ruby
  2. Enable Sidekiq Pro metric collection by including this in your initializer:

        require 'datadog/statsd' # gem 'dogstatsd-ruby'
    
        Sidekiq::Pro.dogstatsd = ->{ Datadog::Statsd.new('metrics.example.com', 8125, namespace:'sidekiq') }
    
        Sidekiq.configure_server do |config|
          config.server_middleware do |chain|
            require 'sidekiq/middleware/server/statsd'
            chain.add Sidekiq::Middleware::Server::Statsd
          end
        end

    If you are using Sidekiq Enterprise and would like to collect historical metrics, include this line as well:

          Sidekiq.configure_server do |config|
            # history is captured every 30 seconds by default
            config.retain_history(30)
          end

    See the Sidekiq Pro and Enterprise documentation for more information, and the Dogstatsd Ruby documentation for further configuration options.

  3. Update the Datadog Agent main configuration file datadog.yaml by adding the following configs:

    # dogstatsd_mapper_cache_size: 1000  # default to 1000
    dogstatsd_mapper_profiles:
     - name: sidekiq
       prefix: "sidekiq."
       mappings:
         - match: 'sidekiq\.sidekiq\.(.*)'
           match_type: "regex"
           name: "sidekiq.$1"
         - match: 'sidekiq\.jobs\.(.*)\.perform'
           name: "sidekiq.jobs.perform"
           match_type: "regex"
           tags:
             worker: "$1"
         - match: 'sidekiq\.jobs\.(.*)\.(count|success|failure)'
           name: "sidekiq.jobs.worker.$2"
           match_type: "regex"
           tags:
             worker: "$1"
  4. Restart the Agent.

Validation

Run the Agent’s status subcommand and look for sidekiq under the Checks section.

Data Collected

Metrics

sidekiq.jobs.perform.count
(gauge)
Count of the amount of time spent in a worker
sidekiq.jobs.perform.avg
(gauge)
Average amount of time spent in a worker
sidekiq.jobs.perform.max
(gauge)
Max amount of time spent in a worker
sidekiq.jobs.perform.median
(gauge)
Median amount of time spent in a worker
sidekiq.jobs.perform.95percentile
(gauge)
95th percentile of amount of time spent in a worker
sidekiq.jobs.count
(count)
Total count of Sidekiq jobs
Shown as job
sidekiq.jobs.success
(count)
Total count of successful Sidekiq jobs
Shown as job
sidekiq.jobs.failure
(count)
Total count of failed Sidekiq jobs
Shown as job
sidekiq.jobs.worker.count
(count)
Count of Sidekiq jobs
Shown as job
sidekiq.jobs.worker.success
(count)
Count of successful Sidekiq jobs
Shown as job
sidekiq.jobs.worker.failure
(count)
Count of failed Sidekiq jobs
Shown as job
sidekiq.processed
(count)
Number of job executions completed (success or failure) (Enterprise only)
Shown as job
sidekiq.failed
(count)
Number of job executions which raised an error (Enterprise only)
Shown as job
sidekiq.enqueued
(count)
Total size of all known queues (Enterprise only)
Shown as job
sidekiq.retries
(count)
Total retries size (Enterprise only)
Shown as job
sidekiq.dead
(count)
Total Dead Size (Enterprise only)
Shown as job
sidekiq.scheduled
(count)
Total Scheduled Size (Enterprise only)
Shown as job
sidekiq.busy
(count)
Total Busy Size (Enterprise only)
Shown as job
sidekiq.jobs.expired
(count)
Count of when a job is expired
Shown as job
sidekiq.jobs.recovered.push
(count)
Count of when a job is recovered by reliable_push after network outage
Shown as job
sidekiq.jobs.recovered.fetch
(count)
Count of when a job is recovered by super_fetch after process crash
Shown as job
sidekiq.batch.created
(count)
Count of when a batch is created
sidekiq.batch.complete
(count)
Count of when a batch is completed
sidekiq.batch.success
(count)
Count of when a batch is successful

The Sidekiq integration also allows custom metrics, see the Sidekiq documentation for custom metric inspiration.

Log Collection

  1. Collecting logs is disabled by default in the Datadog Agent. Enable it in the datadog.yaml file with:

      logs_enabled: true
  2. Add this configuration block to your sidekiq.d/conf.yaml file to start collecting your Sidekiq logs:

      logs:
        - type: file
          path:  /var/log/sidekiq.log
          source: sidekiq
          service: <SERVICE>

    Change the path and service parameter values and configure them for your environment. If you cannot find your logs, see the Sidekiq documentation on more details about logs.

  3. Restart the Agent.

Service Checks

Sidekiq does not include any service checks.

Events

Sidekiq does not include any events.

Troubleshooting

Need help? Contact Datadog support.