---
title: External Sources
description: Connect tables from your own data platform to search them with Federated Logs.
breadcrumbs: Docs > Log Management > Log Configuration > External Sources
---

> For the complete documentation index, see [llms.txt](https://docs.datadoghq.com/llms.txt).

# External Sources

{% callout %}
##### Join the Preview!

Federated Logs is in Preview. Use this form to submit your request today.

[Request Access](https://www.datadoghq.com/product-preview/federated-search/)
{% /callout %}

## Overview{% #overview %}

An external source is a table that contains logs in your own data platform, such as Databricks. After you connect it, you can search it from Log Explorer with [Federated Logs](https://docs.datadoghq.com/logs/explorer/federated_logs.md), without ingesting the logs into Datadog.

## Prerequisites{% #prerequisites %}

- Access to the Federated Logs Preview. To request access, complete the [Preview form](https://www.datadoghq.com/product-preview/federated-search/).
- The Logs Federated Query Write (`logs_federated_query_write`) permission, which lets you connect, edit, and delete sources. For more information, see [Role Based Access Control](https://docs.datadoghq.com/account_management/rbac.md).

## Set up your source{% #set-up-your-source %}

{% tab title="Databricks" %}
### Databricks prerequisites{% #databricks-prerequisites %}

- A [Databricks integration](https://docs.datadoghq.com/integrations/databricks.md) account configured in Datadog. Note the application (client) ID of the service principal used by the integration.
- A Databricks table, registered in Unity Catalog, that contains your logs. You can load logs into the table with any tool, such as the [Observability Pipelines Databricks destination](https://docs.datadoghq.com/observability_pipelines/destinations/databricks.md).

### Grant access to the service principal{% #grant-access-to-the-service-principal %}

Datadog runs queries with the Databricks SQL Statement Execution API, using the service principal of your Databricks integration. Use the service principal's application (client) ID, not its display name, for `<application_id>`.

1. In your Databricks workspace settings, open the service principal, go to the Configurations tab, and select Databricks SQL access. New service principals don't inherit this entitlement from the `users` group. For more information, see [Manage entitlements](https://docs.databricks.com/aws/en/security/auth/entitlements) in the Databricks documentation.

1. Give the service principal the `CAN USE` permission on the SQL warehouse configured in your Databricks integration. Datadog runs queries on this warehouse. `CAN MANAGE` is not required.

1. Grant read access to the table. In Databricks, open the SQL Editor and run the following commands:

   ```sql
   GRANT USE CATALOG ON CATALOG <catalog_name> TO `<application_id>`;
   GRANT USE SCHEMA ON SCHEMA <catalog_name>.<schema_name> TO `<application_id>`;
   GRANT SELECT ON TABLE <catalog_name>.<schema_name>.<table_name> TO `<application_id>`;
   ```

The External Sources page lists only the catalogs, schemas, and tables that the service principal has access to. Queries consume compute on your SQL warehouse.

### Supported column types{% #supported-column-types %}

| Datadog field                  | Supported column types                                   |
| ------------------------------ | -------------------------------------------------------- |
| Timestamp                      | `timestamp`, or `string` in ISO 8601 format              |
| Unique identifier (`_id`)      | `string`                                                 |
| Status                         | `string`                                                 |
| Message, Host, Service, Source | `string`                                                 |
| Tags                           | `array<string>`, with each element in `key:value` format |

Numeric timestamps, such as epoch values, are not supported.

### Source fields{% #source-fields %}

When you connect the source, select Databricks as the source type and your integration account. Then select the catalog, schema, and table that contains your logs.
{% /tab %}

## Schema mapping{% #schema-mapping %}

When you connect a source, you map each Datadog field to a column of your table. Datadog reads each row of the table as one log.

| Datadog field                  | Required |
| ------------------------------ | -------- |
| Timestamp                      | Yes      |
| Unique identifier (`_id`)      | Yes      |
| Status                         | Yes      |
| Message, Host, Service, Source | No       |
| Tags                           | No       |

Only columns with a compatible type are listed for each field. For the supported column types, see the tab for your source in the Set up your source section.

## Connect a source{% #connect-a-source %}

1. Go to [Logs > Configuration > External Sources](https://app.datadoghq.com/logs/external-sources) and click Connect Source.
1. On the Source tab, select the source type and the table that contains your logs. The fields depend on the source. Click Next.
1. On the Schema Mapping tab, select the column to read each Datadog field from. See the Schema mapping section.
1. Click Save.

You can connect multiple tables, each with its own schema mapping. After you save a source, you can edit only its schema mapping. To change the integration account or the table, delete the source and connect it again.

To search an external source, see [Federated Logs](https://docs.datadoghq.com/logs/explorer/federated_logs.md).

## Further reading{% #further-reading %}

Additional helpful documentation, links, and articles:

- [Search external sources with Federated Logs](https://docs.datadoghq.com/logs/explorer/federated_logs.md)
- [Send logs to Databricks with Observability Pipelines](https://docs.datadoghq.com/observability_pipelines/destinations/databricks.md)
