--- title: Fiddler description: Gain visibility into your ML systems with the Fiddler Datadog integration breadcrumbs: Docs > Integrations > Fiddler --- # Fiddler Supported OS Integration version4.0.0 Fiddler Dashboard in DataDogModel Accuracy ChartModel AnalyticsPerforming counterfactual explanationsData Drift ChartsModel Explanations ## Overview{% #overview %} Fiddler's Model Performance Management platform monitors Machine Learning model performance by sending real-time alerts when model performance metrics drop, allowing users to analyze inference data to understand why model performance is degrading. This integration includes metrics and an out-of-the-box dashboard that displays performance metrics such as accuracy, traffic, and drift. ## Setup{% #setup %} The Fiddler check is not included in the [Datadog Agent](https://app.datadoghq.com/account/settings/agent/latest) package, so you need to install it. ### Installation{% #installation %} For Agent v7.21+ / v6.21+, follow the instructions below to install the Fiddler check on your host. See [Use Community Integrations](https://docs.datadoghq.com/agent/guide/use-community-integrations/) to install with the Docker Agent or earlier versions of the Agent. 1. Run the following command to install the Agent integration: ```shell datadog-agent integration install -t datadog-fiddler==4.0.0 ``` 1. Configure your integration similar to Agent-based [integrations](https://docs.datadoghq.com/getting_started/integrations/). ### Configuration{% #configuration %} 1. Edit the `fiddler.d/conf.yaml` file, in the `conf.d/` folder at the root of your Agent's configuration directory to start collecting your Fiddler performance data. See the [sample `fiddler.d/conf.yaml`](https://github.com/DataDog/integrations-extras/blob/master/fiddler/datadog_checks/fiddler/data/conf.yaml.example) for all available configuration options. The `url`, `org`, and `fiddler_api_key` parameters need to be updated for the Fiddler environment you wish the integration to query. Fiddler also suggests setting the `minimum_collection_interval` setting in the `conf.yaml` file to `300` (5 minutes). Additionally, `bin_size`, `v1compat` and `enabled_metrics` can also be configured. 1. [Restart the Agent](https://docs.datadoghq.com/agent/guide/agent-commands/#start-stop-and-restart-the-agent). ### Validation{% #validation %} [Run the Agent's status subcommand](https://docs.datadoghq.com/agent/guide/agent-commands/#agent-status-and-information) and look for `fiddler` under the Checks section. ## Data Collected{% #data-collected %} ### Metrics{% #metrics %} | | | | | **fiddler.accuracy**(gauge) | The accuracy of the model*Shown as percent* | | **fiddler.histogram\_drift**(gauge) | The histogram drift of the model[deprecated]*Shown as percent* | | **fiddler.psi**(gauge) | The PSI of the model*Shown as percent* | | **fiddler.jsd**(gauge) | The JSD of the model*Shown as percent* | | **fiddler.feature\_average**(gauge) | The feature average of the model[deprecated]*Shown as percent* | | **fiddler.output\_average**(gauge) | The output average of the model[deprecated]*Shown as percent* | | **fiddler.average**(gauge) | The feature average of the model*Shown as percent* | | **fiddler.traffic\_count**(gauge) | The traffic count of the model[deprecated]*Shown as percent* | | **fiddler.traffic**(gauge) | The traffic count of the model*Shown as percent* | | **fiddler.fpr**(gauge) | The false positive rate of the model*Shown as percent* | | **fiddler.tpr**(gauge) | The true positive rate of the model[deprecated]*Shown as percent* | | **fiddler.auc**(gauge) | The AUC of the model*Shown as percent* | | **fiddler.f1\_score**(gauge) | The F1 Score of the model*Shown as percent* | | **fiddler.precision**(gauge) | The precision of the model*Shown as percent* | | **fiddler.recall**(gauge) | The recall of the model*Shown as percent* | | **fiddler.mape**(gauge) | The MAPE of the model*Shown as percent* | | **fiddler.wmape**(gauge) | The WMAPE of the model*Shown as percent* | | **fiddler.mae**(gauge) | The MAE of the model*Shown as percent* | | **fiddler.mse**(gauge) | The MSE of the model*Shown as percent* | | **fiddler.r2**(gauge) | The R-squared metric of the model*Shown as percent* | | **fiddler.binary\_cross\_entropy**(gauge) | The binary cross-entropy metric of the model[deprecated]*Shown as percent* | | **fiddler.log\_loss**(gauge) | The log-loss metric of the model*Shown as percent* | | **fiddler.expected\_callibration\_error**(gauge) | The expected calibration error of the model[deprecated]*Shown as percent* | | **fiddler.data\_count**(gauge) | The data count metric of the model*Shown as percent* | | **fiddler.auroc**(gauge) | The AUROC metric of the model*Shown as percent* | | **fiddler.callibrated\_threshold**(gauge) | The calibrated threshold metric of the model[deprecated]*Shown as percent* | | **fiddler.calibrated\_threshold**(gauge) | The calibrated threshold metric of the model*Shown as percent* | | **fiddler.g\_mean**(gauge) | The geometric mean metric of the model[deprecated]*Shown as percent* | | **fiddler.geometric\_mean**(gauge) | The geometric mean metric of the model*Shown as percent* | | **fiddler.sum**(gauge) | The SUM of a feature the model*Shown as percent* | | **fiddler.expected\_calibration\_error**(gauge) | The Expected Callibaration Error of the model*Shown as percent* | ### Service Checks{% #service-checks %} **fiddler.can\_connect** Returns `CRITICAL` if the Agent is unable to connect to and collect metrics from the monitored Fiddler instance. Returns `OK` otherwise. *Statuses: ok, critical* ## Troubleshooting{% #troubleshooting %} Need help? Contact [Fiddler support](https://fiddlerlabs.zendesk.com/hc/en-us).