Gunicorn
Rapport de recherche Datadog : Bilan sur l'adoption de l'informatique sans serveur Rapport : Bilan sur l'adoption de l'informatique sans serveur

Gunicorn

Agent Check Check de l'Agent

Supported OS: Linux Mac OS

Dashboard Gunicorn

Présentation

L’Agent Datadog recueille une métrique clé de Gunicorn : le nombre de processus de travail en cours d’exécution. Il envoie également un check de service, qui vérifie si Gunicorn est en cours d’exécution ou non.

Gunicorn peut fournir directement d’autres métriques via DogStatsD, y compris concernant :

  • Le taux de requêtes total
  • Le taux de requêtes par code de statut (2xx, 3xx, 4xx, 5xx)
  • La durée des requêtes (moyenne, médiane, maximum, 95e centile, etc.)
  • Le taux de message de log par niveau de log (critical, error, warning, exception)

Configuration

Installation

Le check Gunicorn de l’Agent Datadog est inclus avec le paquet de l’Agent Datadog : vous n’avez donc rien d’autre à installer sur vos serveurs Gunicorn.

Le check Gunicorn nécessite l’environnement Python de votre application Gunicorn pour obtenir le package setproctitle. Sans celui-ci, l’Agent Datadog signalera toujours qu’il ne parvient pas à trouver de processus principal gunicorn (et donc, qu’il ne peut pas non plus trouver de workers). Installez le package setproctitle dans l’environnement Python de votre application si vous souhaitez recueillir la métrique gunicorn.workers.

Configuration

Modifiez le fichier gunicorn.d/conf.yaml dans le dossier conf.d/ à la racine du répertoire de configuration de votre Agent pour commencer à recueillir vos métriques et logs Gunicorn. Consultez le fichier d’exemple gunicorn.yaml pour découvrir toutes les options de configuration disponibles.

Collecte de métriques

  1. Ajoutez ce bloc de configuration à votre fichier gunicorn.d/conf.yaml pour commencer à recueillir vos métriques Gunicorn :

    init_config:
    
    instances:
    # tel que défini
    # 1) dans le config.py de votre application (proc_name = <NOM_VOTRE_APPLICATION>), OU
    # 2) via CLI (gunicorn --name <NOM_VOTRE_APPLICATION> your:app)
    - proc_name: <NOM_VOTRE_APPLICATION>
  2. Redémarrez l’Agent pour commencer à envoyer vos métriques Gunicorn à Datadog.

Associer Gunicorn à DogStatsD

Depuis la version 19.1, Gunicorn propose une option pour envoyer ses métriques à un daemon qui implémente le protocole StatsD, tel que DogStatsD. Comme pour de nombreuses options de Gunicorn, vous pouvez la passer à gunicorn via l’interface de ligne de commande (--statsd-host) ou la définir dans le fichier de configuration de votre application (statsd_host). Configurez votre application de façon à envoyer les métriques à DogStatsD sur "localhost:8125", puis redémarrez-la.

Remarque : si vous utilisez cette option, n’ajoutez pas le bloc de configuration pour la collecte de métriques à gunicorn.d/conf.yaml. Ainsi, si vous associez Gunicorn à DogStatsD, ignorez les instructions de la section Collecte de métriques de cette page.

Collecte de logs

Disponible à partir des versions > 6.0 de l’Agent

  1. La collecte de logs est désactivée par défaut dans l’Agent Datadog. Vous devez l’activer dans datadog.yaml :

    logs_enabled: true
  2. Utilisez la commande suivante pour configurer le chemin du fichier de log d’accès comme expliqué dans la documentation Gunicorn (en anglais) : --access-logfile <CHEMIN_DU_FICHIER>

  3. Utilisez la commande suivante pour configurer le chemin du fichier de log d’erreurs comme expliqué dans la documentation Gunicorn (en anglais) : --error-logfile FILE, --log-file <CHEMIN_DU_FICHIER>

  4. Ajoutez ce bloc de configuration à votre fichier gunicorn.d/conf.yaml pour commencer à recueillir vos logs Gunicorn :

    logs:
     - type: file
       path: /var/log/gunicorn/access.log
       service: "<MY_SERVICE>"
       source: gunicorn
    
     - type: file
       path: /var/log/gunicorn/error.log
       service: "<MY_SERVICE>"
       source: gunicorn
       log_processing_rules:
         - type: multi_line
           name: log_start_with_date
           pattern: \[\d{4}-\d{2}-\d{2}

    Modifiez les valeurs des paramètres path et service en fonction de votre environnement. Consultez le fichier d’exemple gunicorn.yaml pour découvrir toutes les options de configuration disponibles.

  5. Redémarrez l’Agent.

Validation

Lancez la sous-commande status de l’Agent et cherchez gunicorn dans la section Checks.

Si le statut n’est pas défini sur OK, consultez la section Dépannage.

Utilisez netstat pour vérifier que Gunicorn envoie également ses propres métriques :

$ sudo netstat -nup | grep "127.0.0.1:8125.*ESTABLISHED"
udp        0      0 127.0.0.1:38374         127.0.0.1:8125          ESTABLISHED 15500/gunicorn: mas

Données collectées

Métriques

gunicorn.requests
(rate)
The rate of requests received.
Shown as request
gunicorn.workers
(gauge)
The number of workers tagged by state (idle or working).
Shown as worker
gunicorn.request.duration.95percentile
(gauge)
The 95th percentile of request duration time.
Shown as millisecond
gunicorn.request.duration.avg
(gauge)
The average request duration time.
Shown as millisecond
gunicorn.request.duration.count
(rate)
The rate of requests received.
Shown as request
gunicorn.request.duration.max
(gauge)
The maximum request duration time.
Shown as millisecond
gunicorn.request.duration.median
(gauge)
The median request duration time.
Shown as millisecond
gunicorn.log.critical
(rate)
The rate of logged critical statements.
Shown as occurrence
gunicorn.log.error
(rate)
The rate of logged errors.
Shown as occurrence
gunicorn.log.warning
(rate)
The rate of logged warnings.
Shown as occurrence
gunicorn.log.exception
(rate)
The rate of logged exceptions.
Shown as occurrence
gunicorn.request.status.100
(rate)
The rate of requests that generate responses with a 100 status code.
Shown as request
gunicorn.request.status.101
(rate)
The rate of requests that generate responses with a 101 status code.
Shown as request
gunicorn.request.status.102
(rate)
The rate of requests that generate responses with a 102 status code.
Shown as request
gunicorn.request.status.200
(rate)
The rate of requests that generate responses with a 200 status code.
Shown as request
gunicorn.request.status.201
(rate)
The rate of requests that generate responses with a 201 status code.
Shown as request
gunicorn.request.status.202
(rate)
The rate of requests that generate responses with a 202 status code.
Shown as request
gunicorn.request.status.203
(rate)
The rate of requests that generate responses with a 203 status code.
Shown as request
gunicorn.request.status.204
(rate)
The rate of requests that generate responses with a 204 status code.
Shown as request
gunicorn.request.status.205
(rate)
The rate of requests that generate responses with a 205 status code.
Shown as request
gunicorn.request.status.206
(rate)
The rate of requests that generate responses with a 206 status code.
Shown as request
gunicorn.request.status.207
(rate)
The rate of requests that generate responses with a 207 status code.
Shown as request
gunicorn.request.status.208
(rate)
The rate of requests that generate responses with a 208 status code.
Shown as request
gunicorn.request.status.226
(rate)
The rate of requests that generate responses with a 226 status code.
Shown as request
gunicorn.request.status.300
(rate)
The rate of requests that generate responses with a 300 status code.
Shown as request
gunicorn.request.status.301
(rate)
The rate of requests that generate responses with a 301 status code.
Shown as request
gunicorn.request.status.302
(rate)
The rate of requests that generate responses with a 302 status code.
Shown as request
gunicorn.request.status.303
(rate)
The rate of requests that generate responses with a 303 status code.
Shown as request
gunicorn.request.status.304
(rate)
The rate of requests that generate responses with a 304 status code.
Shown as request
gunicorn.request.status.305
(rate)
The rate of requests that generate responses with a 305 status code.
Shown as request
gunicorn.request.status.307
(rate)
The rate of requests that generate responses with a 307 status code.
Shown as request
gunicorn.request.status.308
(rate)
The rate of requests that generate responses with a 308 status code.
Shown as request
gunicorn.request.status.400
(rate)
The rate of requests that generate responses with a 400 status code.
Shown as request
gunicorn.request.status.401
(rate)
The rate of requests that generate responses with a 401 status code.
Shown as request
gunicorn.request.status.402
(rate)
The rate of requests that generate responses with a 402 status code.
Shown as request
gunicorn.request.status.403
(rate)
The rate of requests that generate responses with a 403 status code.
Shown as request
gunicorn.request.status.404
(rate)
The rate of requests that generate responses with a 404 status code.
Shown as request
gunicorn.request.status.405
(rate)
The rate of requests that generate responses with a 405 status code.
Shown as request
gunicorn.request.status.406
(rate)
The rate of requests that generate responses with a 406 status code.
Shown as request
gunicorn.request.status.407
(rate)
The rate of requests that generate responses with a 407 status code.
Shown as request
gunicorn.request.status.408
(rate)
The rate of requests that generate responses with a 408 status code.
Shown as request
gunicorn.request.status.409
(rate)
The rate of requests that generate responses with a 409 status code.
Shown as request
gunicorn.request.status.410
(rate)
The rate of requests that generate responses with a 410 status code.
Shown as request
gunicorn.request.status.411
(rate)
The rate of requests that generate responses with a 411 status code.
Shown as request
gunicorn.request.status.412
(rate)
The rate of requests that generate responses with a 412 status code.
Shown as request
gunicorn.request.status.413
(rate)
The rate of requests that generate responses with a 413 status code.
Shown as request
gunicorn.request.status.414
(rate)
The rate of requests that generate responses with a 414 status code.
Shown as request
gunicorn.request.status.415
(rate)
The rate of requests that generate responses with a 415 status code.
Shown as request
gunicorn.request.status.416
(rate)
The rate of requests that generate responses with a 416 status code.
Shown as request
gunicorn.request.status.417
(rate)
The rate of requests that generate responses with a 417 status code.
Shown as request
gunicorn.request.status.419
(rate)
The rate of requests that generate responses with a 419 status code.
Shown as request
gunicorn.request.status.421
(rate)
The rate of requests that generate responses with a 421 status code.
Shown as request
gunicorn.request.status.422
(rate)
The rate of requests that generate responses with a 422 status code.
Shown as request
gunicorn.request.status.423
(rate)
The rate of requests that generate responses with a 423 status code.
Shown as request
gunicorn.request.status.424
(rate)
The rate of requests that generate responses with a 424 status code.
Shown as request
gunicorn.request.status.426
(rate)
The rate of requests that generate responses with a 426 status code.
Shown as request
gunicorn.request.status.428
(rate)
The rate of requests that generate responses with a 428 status code.
Shown as request
gunicorn.request.status.429
(rate)
The rate of requests that generate responses with a 429 status code.
Shown as request
gunicorn.request.status.431
(rate)
The rate of requests that generate responses with a 431 status code.
Shown as request
gunicorn.request.status.451
(rate)
The rate of requests that generate responses with a 451 status code.
Shown as request
gunicorn.request.status.500
(rate)
The rate of requests that generate responses with a 500 status code.
Shown as request
gunicorn.request.status.501
(rate)
The rate of requests that generate responses with a 501 status code.
Shown as request
gunicorn.request.status.502
(rate)
The rate of requests that generate responses with a 502 status code.
Shown as request
gunicorn.request.status.503
(rate)
The rate of requests that generate responses with a 503 status code.
Shown as request
gunicorn.request.status.504
(rate)
The rate of requests that generate responses with a 504 status code.
Shown as request
gunicorn.request.status.505
(rate)
The rate of requests that generate responses with a 505 status code.
Shown as request
gunicorn.request.status.506
(rate)
The rate of requests that generate responses with a 506 status code.
Shown as request
gunicorn.request.status.507
(rate)
The rate of requests that generate responses with a 507 status code.
Shown as request
gunicorn.request.status.508
(rate)
The rate of requests that generate responses with a 508 status code.
Shown as request
gunicorn.request.status.510
(rate)
The rate of requests that generate responses with a 510 status code.
Shown as request
gunicorn.request.status.511
(rate)
The rate of requests that generate responses with a 511 status code.
Shown as request
gunicorn.request.status.512
(rate)
The rate of requests that generate responses with a 512 status code.
Shown as request

Événements

Le check Gunicorn n’inclut aucun événement.

Checks de service

gunicorn.is_running :
Renvoie CRITICAL si l’Agent ne parvient pas à trouver le processus maître Gunicorn ou s’il ne trouve aucun worker, actif ou non. Si ce n’est pas le cas, renvoie OK.

Dépannage

L’Agent ne trouve pas de processus Gunicorn

  Checks
  ======

    gunicorn (5.12.1)
    -----------------
      - instance #0 [ERROR]: 'Found no master process with name: gunicorn: master [my_web_app]'
      - Collected 0 metrics, 0 events & 1 service check
      - Dependencies:
          - psutil: 4.4.1

Soit Gunicorn ne s’exécute pas réellement, soit le paquet setproctitle n’est pas installé dans l’environnement Python de votre application.

Si setproctitle n’est pas installé, Gunicorn apparaît dans la table des processus comme suit :

$ ps -ef | grep gunicorn
ubuntu   18013 16695  2 20:23 pts/0    00:00:00 /usr/bin/python /usr/bin/gunicorn --config test-app-config.py gunicorn-test:app
ubuntu   18018 18013  0 20:23 pts/0    00:00:00 /usr/bin/python /usr/bin/gunicorn --config test-app-config.py gunicorn-test:app
ubuntu   18019 18013  0 20:23 pts/0    00:00:00 /usr/bin/python /usr/bin/gunicorn --config test-app-config.py gunicorn-test:app

S’il est bien installé, les processus gunicorn apparaissent au format attendu par l’Agent Datadog :

$ ps -ef | grep gunicorn
ubuntu   18457 16695  5 20:26 pts/0    00:00:00 gunicorn: master [my_app]
ubuntu   18462 18457  0 20:26 pts/0    00:00:00 gunicorn: worker [my_app]
ubuntu   18463 18457  0 20:26 pts/0    00:00:00 gunicorn: worker [my_app]

Pour aller plus loin