Supported OS Linux Windows Mac OS

Versión de la integración1.0.0
This page is not yet available in Spanish. We are working on its translation.
If you have any questions or feedback about our current translation project, feel free to reach out to us!

Overview

This check monitors Celery through the Datadog Agent. Celery is a distributed task queue system that enables asynchronous task processing in Python applications.

The Celery integration provides valuable insights into your task queue system by:

  • Monitoring worker health, status, and task execution metrics
  • Tracking task processing rates, runtime, and prefetch times
  • Providing visibility into worker performance and task distribution
  • Helping identify bottlenecks and optimize task processing efficiency

Setup

Follow the instructions below to install and configure this check for an Agent running on a host. For containerized environments, see the Autodiscovery Integration Templates for guidance on applying these instructions.

Installation

The Celery check is included in the Datadog Agent package. No additional installation is needed on your server.

Prerequisites

  1. Install and configure Celery Flower, the real-time web monitor and administration tool for Celery.

Configuration

  1. Edit the celery.d/conf.yaml file in the conf.d/ folder at the root of your Agent’s configuration directory to start collecting your Celery performance data. See the sample celery.d/conf.yaml for all available configuration options.

    init_config:
    
    instances:
      ## @param openmetrics_endpoint - string - required
      ## Endpoint exposing the Celery Flower's Prometheus metrics
      #
      - openmetrics_endpoint: http://localhost:5555/metrics
    
  2. Restart the Agent.

Validation

Run the Agent’s status subcommand and look for celery under the Checks section.

Data Collected

Metrics

celery.flower.events.count
(count)
The count of Celery events from the last submission.
Shown as event
celery.flower.events.created
(gauge)
The number of Celery events created.
Shown as event
celery.flower.task.prefetch_time.seconds
(gauge)
Time tasks spend waiting at worker before execution.
Shown as second
celery.flower.task.runtime.created
(gauge)
Task runtime creation timestamp.
Shown as second
celery.flower.task.runtime.seconds.bucket
(count)
The number of observations within each distribution bucket of the tasks runtime.
celery.flower.task.runtime.seconds.count
(count)
Task runtime duration.
Shown as second
celery.flower.task.runtime.seconds.sum
(count)
The total duration of a task runtime.
Shown as second
celery.flower.worker.executing_tasks
(gauge)
Number of tasks currently executing at a worker.
Shown as task
celery.flower.worker.online
(gauge)
Worker online status (1 for online, 0 for offline).
celery.flower.worker.prefetched_tasks
(gauge)
Number of tasks prefetched at a worker.
Shown as task

Events

The Celery integration does not include any events.

Service Checks

celery.flower.openmetrics.health
Returns CRITICAL if the Agent is unable to connect to the Celery OpenMetrics endpoint, otherwise returns OK.
Statuses: ok, critical

Troubleshooting

Need help? Contact Datadog support.