--- title: TiDB description: The integration for TiDB cluster breadcrumbs: Docs > Integrations > TiDB --- # TiDB Supported OS Integration version2.1.1 ## Overview{% #overview %} Connect [TiDB](https://docs.pingcap.com/tidb/stable) cluster to Datadog in order to: - Collect key TiDB metrics of your cluster. - Collect logs of your cluster, such as TiDB/TiKV/TiFlash logs and slow query logs. - Visualize cluster performance on the provided dashboard. **Note**: - TiDB 4.0+ is required for this integration. - For TiDB Cloud, see the [TiDB Cloud Integration](https://docs.datadoghq.com/integrations/tidb_cloud/). ## Setup{% #setup %} ### Installation{% #installation %} First, [download and launch the Datadog Agent](https://app.datadoghq.com/account/settings/agent/latest). Then, manually install the TiDB check. [Instructions vary depending on the environment](https://docs.datadoghq.com/agent/guide/community-integrations-installation-with-docker-agent). Run `datadog-agent integration install -t datadog-tidb==`. ### Configuration{% #configuration %} ##### Metric collection{% #metric-collection %} 1. Edit the `tidb.d/conf.yaml` file in the `conf.d/` folder at the root of your Agent's configuration directory to start collecting your TiDB performance data. See the [sample tidb.d/conf.yaml](https://github.com/DataDog/integrations-extras/blob/master/tidb/datadog_checks/tidb/data/conf.yaml.example) for all available configuration options. The [sample tidb.d/conf.yaml](https://github.com/DataDog/integrations-extras/blob/master/tidb/datadog_checks/tidb/data/conf.yaml.example) only configures the PD instance. You need to manually configure the other instances in the TiDB cluster. Like this: ```yaml init_config: instances: - pd_metric_url: http://localhost:2379/metrics send_distribution_buckets: true tags: - cluster_name:cluster01 - tidb_metric_url: http://localhost:10080/metrics send_distribution_buckets: true tags: - cluster_name:cluster01 - tikv_metric_url: http://localhost:20180/metrics send_distribution_buckets: true tags: - cluster_name:cluster01 - tiflash_metric_url: http://localhost:8234/metrics send_distribution_buckets: true tags: - cluster_name:cluster01 - tiflash_proxy_metric_url: http://localhost:20292/metrics send_distribution_buckets: true tags: - cluster_name:cluster01 ``` [Restart the Agent](https://docs.datadoghq.com/agent/guide/agent-commands/?tab=agentv6v7#restart-the-agent). ##### Log collection{% #log-collection %} *Available for Agent versions >6.0* 1. Collecting logs is disabled by default in the Datadog Agent, enable it in your `datadog.yaml` file: ```yaml logs_enabled: true ``` 1. Add this configuration block to your `tidb.d/conf.yaml` file to start collecting your TiDB logs: ```yaml logs: # pd log - type: file path: "/tidb-deploy/pd-2379/log/pd*.log" service: "tidb-cluster" source: "pd" # tikv log - type: file path: "/tidb-deploy/tikv-20160/log/tikv*.log" service: "tidb-cluster" source: "tikv" # tidb log - type: file path: "/tidb-deploy/tidb-4000/log/tidb*.log" service: "tidb-cluster" source: "tidb" exclude_paths: - /tidb-deploy/tidb-4000/log/tidb_slow_query.log - type: file path: "/tidb-deploy/tidb-4000/log/tidb_slow_query*.log" service: "tidb-cluster" source: "tidb" log_processing_rules: - type: multi_line name: new_log_start_with_datetime pattern: '#\sTime:' tags: - "custom_format:tidb_slow_query" # tiflash log - type: file path: "/tidb-deploy/tiflash-9000/log/tiflash*.log" service: "tidb-cluster" source: "tiflash" ``` Change the `path` and `service` according to your cluster's configuration. Use these commands to show all log path: ```shell # show deploying directories tiup cluster display # find specific logging file path by command arguments ps -fwwp ``` 1. [Restart the Agent](https://docs.datadoghq.com/agent/guide/agent-commands/?tab=agentv6v7#restart-the-agent). ### Validation{% #validation %} Run the [Agent's status subcommand](https://docs.datadoghq.com/agent/guide/agent-commands/#agent-status-and-information) and look for `tidb` under the Checks section. ## Data Collected{% #data-collected %} ### Metrics{% #metrics %} | | | | | **tidb\_cluster.tidb\_executor\_statement\_total**(count) | The total number of statements executed*Shown as execution* | | **tidb\_cluster.tidb\_server\_execute\_error\_total**(count) | The total number of execution errors*Shown as error* | | **tidb\_cluster.tidb\_server\_connections**(gauge) | Current number of connections in TiDB server*Shown as connection* | | **tidb\_cluster.tidb\_server\_handle\_query\_duration\_seconds.count**(count) | The total number of handled queries in server*Shown as query* | | **tidb\_cluster.tidb\_server\_handle\_query\_duration\_seconds.sum**(count) | The sum of handled query duration in server*Shown as second* | | **tidb\_cluster.tikv\_engine\_size\_bytes**(gauge) | The disk usage bytes of TiKV instances*Shown as byte* | | **tidb\_cluster.tikv\_store\_size\_bytes**(gauge) | The disk capacity bytes of TiKV instances*Shown as byte* | | **tidb\_cluster.tikv\_io\_bytes**(count) | The io read/write bytes of TiKV instances*Shown as byte* | | **tidb\_cluster.tiflash\_store\_size\_used\_bytes**(gauge) | The disk usage bytes of TiFlash instances*Shown as byte* | | **tidb\_cluster.tiflash\_store\_size\_capacity\_bytes**(gauge) | The disk capacity bytes of TiFlash instances*Shown as byte* | | **tidb\_cluster.process\_cpu\_seconds\_total**(count) | The cpu usage seconds of TiDB/TiKV/TiFlash instances*Shown as second* | | **tidb\_cluster.process\_resident\_memory\_bytes**(gauge) | The resident memory bytes of TiDB/TiKV/TiFlash instances*Shown as byte* | It is possible to use the `metrics` configuration option to collect additional metrics from a TiDB cluster. ### Events{% #events %} TiDB check does not include any events. ### Service Checks{% #service-checks %} Service Checks are based on `tidb_cluster.prometheus.health` metrics. This check is controlled by the `health_service_check` config and default to `true`. You can modify this behavior in `tidb.yml` file. **tidb\_cluster.prometheus.health** Returns `CRITICAL` if the Agent cannot fetch Prometheus metrics, otherwise returns `OK`. *Statuses: ok, critical* ## Troubleshooting{% #troubleshooting %} ### Missing CPU and Memory metrics for TiKV and TiFlash instances on macOS{% #missing-cpu-and-memory-metrics-for-tikv-and-tiflash-instances-on-macos %} CPU and Memory metrics are not provided for TiKV and TiFlash instances in the following cases: - Running TiKV or TiFlash instances with [tiup playground](https://docs.pingcap.com/tidb/stable/tiup-playground) on macOS. - Running TiKV or TiFlash instances with [docker-compose up](https://github.com/DataDog/integrations-extras/tree/master/tidb/tests/compose) on a new Apple M1 machine. ### Too many metrics{% #too-many-metrics %} The TiDB check enables Datadog's `distribution` metric type by default. This part of data is quite large and may consume lots of resources. You can modify this behavior in `tidb.yml` file: - `send_distribution_buckets: false` Since there are many important metrics in a TiDB cluster, the TiDB check sets `max_returned_metrics` to `10000` by default. You can decrease `max_returned_metrics` in `tidb.yml` file if necessary: - `max_returned_metrics: 1000` Need help? Contact [Datadog support](https://docs.datadoghq.com/help/).