Messages feature allows identifying the root cause of poison pill messages and to better understand data streams by inspecting message content. It allows viewing Kafka messages at specific partitions and offsets.

Supported data formats

Avro, Protobuf, and JSON are supported.

Prerequisites

Kafka Consumer integration

Kafka Consumer integration needs to be set up on any consumer consuming from the topic you want to retrieve messages from. If a topic has more than one consumer group, setting up the integration on one of these consumer groups is enough to use the feature.

Validation

Ensure that the Kafka Consumer check is running correctly by following these instructions

Agent version

Ensure the agent version you are running is 7.70 or later.

Validation

1. Run the Agent’s status subcommand and check the agent version.
2. In Datadog, under integrations, View agents, find the agent running the Kafka Consumer integration, and check its version.

Remote configuration

Ensure remote configuration is set up for the agent running the Kafka Consumer integration.

Validation

1. In Datadog, under Remote Configuration, check that remote configuration is enabled at the organization level.
2. In Datadog, under Remote Configuration, check that the agent running the Kafka Consumer integration has remote configuration enabled, and is using an API key with remote configuration enabled.

Required permissions

You must have the Data Streams Monitoring Capture Messages permission, and these logs permissions that are part of the Datadog Standard role:

• Logs Read Index Data
• Logs Read Data
• Logs Live Tail

You can verify your current permissions on your Profile page. To enable permissions, edit an existing role or create a new one on the Roles page. If you do not have permission to modify roles, contact your organization’s administrator.

1. Create a new role

1. Navigate to the Roles page in Datadog.
2. Click + New Role in the top-right corner.
   If you see "Read Only" instead of the "+ New Role button", you don't have permission to create roles. Contact your Datadog administrator for assistance.
3. Enter a descriptive name for your new role (for example, “Data Streams Messages Access”).
4. In the Search Permissions field, type Data Streams Monitoring Capture Messages.
5. Select the permission from the search results to enable it for this role.
6. Click Save.
7. Confirm your role was created successfully by searching for it in the roles list.

2. Assign the role to users

1. Go to the Users page in Datadog.
2. Find and click on the user you want to assign the role to.
3. In the user details panel, click Edit next to their name.
   If you don't see an "Edit" button, you need administrator privileges to modify user roles. Contact your Datadog administrator.
4. In the modal that opens, locate the Roles section.
5. Add your newly created role to the user.
6. Click Save.
7. Look for a “User updated” confirmation message to verify the change was successful.