--- title: Setup Data Streams Monitoring for .NET description: Datadog, the leading service for cloud-scale monitoring. breadcrumbs: >- Docs > Data Streams Monitoring > Setup Data Streams Monitoring > Setup Data Streams Monitoring for .NET --- # Setup Data Streams Monitoring for .NET {% callout %} # Important note for users on the following Datadog sites: app.ddog-gov.com {% alert level="danger" %} This product is not supported for your selected [Datadog site](https://docs.datadoghq.com/getting_started/site). (). {% /alert %} {% /callout %} ### Prerequisites{% #prerequisites %} - [Datadog Agent v7.34.0 or later](https://docs.datadoghq.com/agent) ### Supported libraries{% #supported-libraries %} | Technology | Library | Minimal tracer version | Recommended tracer version | | -------------------------------------------- | --------------------------------------------------------------------------------- | ---------------------- | -------------------------- | | Kafka | [Confluent.Kafka](https://www.nuget.org/packages/Confluent.Kafka) | 2.28.0 | 2.41.0 or later | | RabbitMQ | [RabbitMQ.Client](https://www.nuget.org/packages/RabbitMQ.Client) | 2.28.0 | 2.37.0 or later | | Amazon SQS | [Amazon SQS SDK](https://www.nuget.org/packages/AWSSDK.SQS) | 2.48.0 | 2.48.0 or later | | Amazon SNS | [Amazon SNS SDK](https://www.nuget.org/packages/AWSSDK.SimpleNotificationService) | 3.6.0 | 3.6.0 or later | | Amazon Kinesis | [Amazon Kinesis SDK](https://www.nuget.org/packages/AWSSDK.Kinesis) | 3.7.0 | 3.7.0 or later | | IBM MQ | [IBMMQDotnetClient](https://www.nuget.org/packages/IBMMQDotnetClient) | 2.49.0 | 2.49.0 or later | | Azure Service Bus(requires additional setup) | [Azure.Messaging.ServiceBus](https://docs.datadoghq.com/agent) | 2.53.0 | 2.53.0 or later | ### Installation{% #installation %} .NET uses auto-instrumentation to inject and extract additional metadata required by Data Streams Monitoring for measuring end-to-end latencies and the relationship between queues and services. {% tab title=".NET Tracer >= v3.22.0 (Recommended)" %} Starting with version 3.22.0 of the .NET tracer, Data Streams Monitoring is in a default-enabled state. Applications with the APM tracer automatically send DSM telemetry, allowing teams to try DSM without an added instrumentation step. If your organization has APM Enterprise, APM Pro or DSM in the contract, the data is processed and stored, enabling DSM views and metrics automatically. When `DD_DATA_STREAMS_ENABLED` is not set, then: - Schema tracking is disabled. - Data Streams is not enabled when running in a serverless environment. - Data Streams information is not propagated for certain messages which are too small or too large. See Message sizes for more details. - Message sizes are not tracked. When `DD_DATA_STREAMS_ENABLED` is set to `true`, then: - Schema tracking is enabled. - Data Streams is enabled for serverless environments. - Data Streams information is sent for **all** messages. - Message sizes are tracked. When `DD_DATA_STREAMS_ENABLED` is set to `false`, then all Data Streams Manager functionality is disabled. If you have any questions regarding default-enabled behavior, reach out to your Customer Success Manager. {% /tab %} {% tab title=".NET Tracer < v3.22.0 (Legacy)" %} To enable Data Streams Monitoring, set the `DD_DATA_STREAMS_ENABLED` environment variable to `true` on services sending messages to (or consuming messages from) your streaming applications. For example: ```yaml environment: - DD_DATA_STREAMS_ENABLED: "true" - DD_TRACE_REMOVE_INTEGRATION_SERVICE_NAMES_ENABLED: "true" ``` {% /tab %} ### Monitoring Kafka Pipelines{% #monitoring-kafka-pipelines %} Data Streams Monitoring uses message headers to propagate context through Kafka streams. If `log.message.format.version` is set in the Kafka broker configuration, it must be set to `0.11.0.0` or higher. Data Streams Monitoring is not supported for versions lower than this. ### Monitoring SQS pipelines{% #monitoring-sqs-pipelines %} Data Streams Monitoring uses one [message attribute](https://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/sqs-message-metadata.html) to track a message's path through an SQS queue. As Amazon SQS has a maximum limit of 10 message attributes allowed per message, all messages streamed through the data pipelines must have 9 or fewer message attributes set, allowing the remaining attribute for Data Streams Monitoring. ### Monitoring RabbitMQ pipelines{% #monitoring-rabbitmq-pipelines %} The [RabbitMQ integration](https://docs.datadoghq.com/integrations/rabbitmq/?tab=host) can provide detailed monitoring and metrics of your RabbitMQ deployments. For full compatibility with Data Streams Monitoring, Datadog recommends configuring the integration as follows: ```yaml instances: - prometheus_plugin: url: http://:15692 unaggregated_endpoint: detailed?family=queue_coarse_metrics&family=queue_consumer_count&family=channel_exchange_metrics&family=channel_queue_exchange_metrics&family=node_coarse_metrics ``` This ensures that all RabbitMQ graphs populate, and that you see detailed metrics for individual exchanges as well as queues. ### Monitoring SNS-to-SQS pipelines{% #monitoring-sns-to-sqs-pipelines %} To monitor a data pipeline where Amazon SNS talks directly to Amazon SQS, you must enable [Amazon SNS raw message delivery](https://docs.aws.amazon.com/sns/latest/dg/sns-large-payload-raw-message-delivery.html). ### Monitoring Azure Service Bus{% #monitoring-azure-service-bus %} Setting up Data Streams Monitoring for Azure Service Bus applications requires additional configuration for the instrumented application. 1. Either set the environment variable `AZURE_EXPERIMENTAL_ENABLE_ACTIVITY_SOURCE` to `true`, or in your application code set the `Azure.Experimental.EnableActivitySource` context switch to `true`. This instructs the Azure Service Bus library to generate tracing information. See [Azure SDK documentation](https://github.com/Azure/azure-sdk-for-net/blob/main/sdk/core/Azure.Core/samples/Diagnostics.md#enabling-experimental-tracing-features) for more details. 1. Set the `DD_TRACE_OTEL_ENABLED` environment variable to `true`. This instructs the .NET auto-instrumentation to listen to the tracing information generated by the Azure Service Bus Library and enables the inject and extract operations required for Data Streams Monitoring. ### Monitoring connectors{% #monitoring-connectors %} #### Confluent Cloud connectors{% #confluent-cloud-connectors %} Data Streams Monitoring can automatically discover your [Confluent Cloud](https://docs.datadoghq.com/integrations/rabbitmq/?tab=host) connectors and visualize them within the context of your end-to-end streaming data pipeline. ##### Setup{% #setup %} 1. Install and configure the [Datadog-Confluent Cloud integration](https://app.datadoghq.com/integrations/confluent-cloud). 1. In Datadog, open the [Confluent Cloud integration tile](https://app.datadoghq.com/integrations/confluent-cloud). Under **Actions**, a list of resources populates with detected clusters and connectors. Datadog attempts to discover new connectors every time you view this integration tile. 1. Select the resources you want to add. 1. Click **Add Resources**. 1. Navigate to [Data Streams Monitoring](https://app.datadoghq.com/data-streams/) to visualize the connectors and track connector status and throughput. ### Message sizes{% #message-sizes %} When Data Streams Monitoring is enabled in default mode, some messages are not instrumented when they are too small, or too large. The following size thresholds apply when Data Streams Monitoring is enabled in default mode: - **Kafka** - Messages less than 34 bytes are not instrumented by default. - **RabbitMQ** - Messages greater than 128 kilobytes are not instrumented by default. - **Amazon Kinesis** - Messages less than 34 bytes are not instrumented by default. ## Further reading{% #further-reading %} - [Autodiscover Confluent Cloud connectors and easily monitor performance in Data Streams Monitoring](https://www.datadoghq.com/blog/confluent-connector-dsm-autodiscovery/)