marketplaceOverview
WhyLabs is a platform built to help organizations monitor, manage, and optimize Artificial Intelligence (AI) applications. It provides a suite of tools to ensure that machine learning (ML) models remain reliable, transparent, and fair throughout their entire lifecycle. The platform leverages monitoring and observability techniques to track model performance, identify issues like data drift or anomalies, and support teams in maintaining high-quality predictions.
This integration ingests WhyLabs data as logs, metrics, and events in Datadog:
Metrics
| |
|---|
cds.whylabs.dataset_metric.classification_accuracy
(gauge) | Classification Accuracy score |
cds.whylabs.dataset_metric.classification_auc
(gauge) | Classification Macro AUC score |
cds.whylabs.dataset_metric.classification_f1
(gauge) | Classification F1 score |
cds.whylabs.dataset_metric.classification_fpr
(gauge) | Classification FPR score |
cds.whylabs.dataset_metric.classification_precision
(gauge) | Classification Precision score |
cds.whylabs.dataset_metric.classification_prediction_count
(gauge) | Classification Prediction count |
cds.whylabs.dataset_metric.classification_recall
(gauge) | Classification Recall score |
cds.whylabs.dataset_metric.regression_mae
(gauge) | Regression mean absolute error score |
cds.whylabs.dataset_metric.regression_mse
(gauge) | Regression mean squared error score |
cds.whylabs.dataset_metric.regression_prediction_count
(gauge) | Regression Prediction count |
cds.whylabs.dataset_metric.regression_rmse
(gauge) | Regression root mean squared error score |
Logs
- Resources
- Entity Schema
- Anomalies
- Segments
Events
The Datadog integration configuration is validated to ensure all required settings are correctly configured before proceeding, followed by tracking authentication events during data ingestion to ensure secure access and proper user verification after the configuration validation.
Dashboards
This integration includes three out-of-the-box dashboards:
- WhyLabs Overview: Provides a comprehensive view of the platform, allowing you to monitor and manage machine learning models and datasets. It highlights key areas such as resources, anomalies, segments, inputs, outputs, and columns.
- WhyLabs - Models: Focuses on essential elements like the model summary, detected anomalies, segments, active monitors, inputs, and outputs. It offers a detailed view of the model’s performance and behavior in production.
- WhyLabs - Datasets: Displays an overview of the dataset datatype, emphasizing key areas such as the model summary, anomalies, segments, active monitors, columns, and discreteness status.
Support
For support or feature requests, contact Crest Data through the following channels:
This application is made available through the Marketplace and is supported by a Datadog Technology Partner. Click Here to purchase this application.
