Algorithmia is an MLOps platform that includes capabilities for data scientists, application developers, and IT operators to deploy, manage, govern, and secure machine learning and other probabilistic models in production.
Algorithmia Insights is a feature of Algorithmia Enterprise and provides a metrics pipeline that can be used to instrument, measure, and monitor your machine learning models. Use cases for monitoring inference-related metrics from machine learning models include detecting model drift, data drift, model bias, etc.
This integration allows you to stream operational metrics as well as user-defined, inference-related metrics from Algorithmia to Kafka to the metrics API in Datadog.
From your Algorithmia instance, configure Algorithmia Insights to connect to a Kafka broker (external to Algorithmia).
Refer to the documentation in the Algorithmia integrations repository to install, configure, and start the Datadog message forwarding service used in this integration, which will forward metrics from a Kafka topic to the metrics API in Datadog.
This integration streams metrics from Algorithmia when a model that has Insights enabled is queried. Each log entry includes operational metrics and inference-related metrics.
duration_milliseconds metric is one of the operational metrics that is
included in the default payload from Algorithmia. To help you get started, this
integration also includes a dashboard and monitor for this default metric.
Additional metrics can include any user-defined, inference-related metrics that
are specified in Algorithmia by the algorithm developer. User-defined metrics
will depend on your specific machine learning framework and use case, but might
include values such as prediction probabilities from a regression model in
scikit-learn, confidence scores from an image classifier in TensorFlow, or input
data from incoming API requests. Note that the message forwarding script
provided in this integration will also prefix user-defined metrics with
algorithmia. in Datadog.
|Duration of algorithm run|
Shown as millisecond
The Algorithmia check does not include any service checks.
The Algorithmia check does not include any events.
Need help? Contact Algorithmia support.