Note
Access to this page requires authorization. You can try signing in or changing directories.
Access to this page requires authorization. You can try changing directories.
External engines can read Unity Catalog tables with fine-grained access controls enforced. This includes both attribute-based access controls (ABAC) policies and standalone row filters and column masks, all dynamically enforced even when querying from outside Azure Databricks.
When an external engine queries a table with ABAC policies, row filters, or column masks attached, Azure Databricks uses a specialized serverless compute layer to filter and return sanitized data to the external engine.
Requirements
To enforce fine-grained access controls on tables queried from external engines, you must complete the following:
- Enable external data access on your Unity Catalog metastore.
- Grant the querying principal the
EXTERNAL USE SCHEMAprivilege. - Use a managed table with catalog commits.
- Authenticate using OAuth machine-to-machine (M2M) or a personal access token (PAT).
Create a managed Delta table with catalog commits
To create a new managed Delta table with catalog commits (requires Databricks Runtime 16.4 and above):
CREATE TABLE <catalog>.<schema>.<table> (id INT, name STRING)
USING delta
TBLPROPERTIES ('delta.feature.catalogManaged' = 'supported');
To upgrade an existing managed table (requires Databricks Runtime 18.0 and above):
ALTER TABLE <catalog>.<schema>.<table>
SET TBLPROPERTIES ('delta.feature.catalogManaged' = 'supported');
After you create the table, you can apply ABAC policies, row filters, or column masks.
See Create a policy or Manually apply row filters and column masks.
Read tables with Apache Spark (Delta)
Configure Apache Spark with Delta-Spark 4.1 or above and Unity Catalog Spark connector 0.4 or above.
"spark.sql.extensions": "io.delta.sql.DeltaSparkSessionExtension",
"spark.sql.catalog.spark_catalog": "org.apache.spark.sql.delta.catalog.DeltaCatalog",
"spark.sql.catalog.<uc-catalog-name>": "io.unitycatalog.spark.UCSingleCatalog",
"spark.sql.catalog.<uc-catalog-name>.uri": "<workspace-url>",
"spark.sql.catalog.<uc-catalog-name>.auth.type": "oauth",
"spark.sql.catalog.<uc-catalog-name>.auth.oauth.uri": "<oauth-token-endpoint>",
"spark.sql.catalog.<uc-catalog-name>.auth.oauth.clientId": "<oauth-client-id>",
"spark.sql.catalog.<uc-catalog-name>.auth.oauth.clientSecret": "<oauth-client-secret>",
"spark.sql.catalog.<uc-catalog-name>.serverSidePlanning.enabled": "true",
"spark.sql.defaultCatalog": "<uc-catalog-name>",
"spark.jars.packages": "io.delta:delta-spark_4.0_2.13:4.1.0,io.delta:delta-iceberg_2.13:4.1.0,io.unitycatalog:unitycatalog-spark_2.13:0.4.0,org.apache.hadoop:hadoop-azure:3.4.2"
Note
Set serverSidePlanning.enabled to true to enable fine-grained access control enforcement from external engines.
Substitute the following variables:
<uc-catalog-name>: The name of the catalog in Unity Catalog that contains your tables.<workspace-url>: The Azure Databricks workspace URL, including the workspace ID.<oauth-token-endpoint>: OAuth token endpoint URL. See Authorize service principal access to Azure Databricks with OAuth.<oauth-client-id>: OAuth client ID for the authenticating principal.<oauth-client-secret>: OAuth client secret for the authenticating principal.
Read tables with Apache Spark (Iceberg)
Configure Apache Spark with Iceberg-Spark 1.11 or above and Apache Spark 4.0 or above.
"spark.sql.extensions": "org.apache.iceberg.spark.extensions.IcebergSparkSessionExtensions",
"spark.sql.catalog.<uc-catalog-name>": "org.apache.iceberg.spark.SparkCatalog",
"spark.sql.catalog.<uc-catalog-name>.type": "rest",
"spark.sql.catalog.<uc-catalog-name>.uri": "<workspace-url>/api/2.1/unity-catalog/iceberg-rest/",
"spark.sql.catalog.<uc-catalog-name>.credential": "<oauth-client-id>:<oauth-client-secret>",
"spark.sql.catalog.<uc-catalog-name>.oauth2-server-uri": "<oauth-token-endpoint>",
"spark.sql.catalog.<uc-catalog-name>.warehouse": "<uc-catalog-name>",
"spark.sql.catalog.<uc-catalog-name>.cache-enabled": "false",
"spark.sql.defaultCatalog": "<uc-catalog-name>"
Substitute the following variables:
<uc-catalog-name>: The name of the catalog in Unity Catalog that contains your tables.<workspace-url>: The Azure Databricks workspace URL, including the workspace ID.<oauth-token-endpoint>: OAuth token endpoint URL. See Authorize service principal access to Azure Databricks with OAuth.<oauth-client-id>: OAuth client ID for the authenticating principal.<oauth-client-secret>: OAuth client secret for the authenticating principal.
Query data
You can query the table using Apache Spark SQL or DataFrame APIs. Azure Databricks enforces fine-grained access policies behind the scenes.
SELECT * FROM <uc-catalog-name>.<schema>.<table>;
Warning
Concurrent writes during query planning can cause the same table to be read from different table snapshots in self-join and multi-scan queries, potentially resulting in incorrect results.
Write to tables with ABAC policies
External engines can write to managed tables with ABAC policies when the authenticated principal is exempted from the policies.
- Add the principal, or an account group containing it, to the
EXCEPTclause (Except for in Catalog Explorer) of every applicable ABAC row filter and column mask policy. Include policies inherited from the metastore, catalog, or schema. See Edit a policy and CREATE POLICY. - Grant the principal
USE CATALOG,USE SCHEMA,EXTERNAL USE SCHEMA, and the table privileges needed to read and write (SELECTandMODIFY). - Configure a supported table for external writes from a Delta client or an Iceberg client.
- Authenticate the external client as the exempt principal.
For example, you run the following statements from an external Iceberg or Delta client to write to and read a managed table where the principal is exempt from all ABAC policies:
INSERT INTO <uc-catalog-name>.<schema>.<table> (id, name)
VALUES (1, 'Example customer');
SELECT * FROM <uc-catalog-name>.<schema>.<table>;
See Access Databricks data using external systems and Attribute-based access control in Unity Catalog.
Serverless compute costs
Cross-engine ABAC uses serverless compute resources to enforce fine-grained access policies server-side. Customers are charged for these resources. For pricing information, see Cross-engine ABAC pricing.
Users with access to the billing system table can query system.billing.usage to see how much they've been charged. For example, the following query breaks down compute costs by user:
SELECT usage_date,
sku_name,
identity_metadata.run_as,
SUM(usage_quantity) AS `DBUs consumed by cross-engine ABAC`
FROM system.billing.usage
WHERE usage_date BETWEEN '2026-06-01' AND '2026-07-01'
AND billing_origin_product = 'EXTERNAL_COMPATIBILITY'
GROUP BY 1, 2, 3 ORDER BY 1;
Limitations
- Dynamic views are not supported.
- Projecting
VARIANTcolumns is not supported. - Filtering on
BINARYcolumns is not supported. - Column-masking functions whose return type differs from the original column type are not supported.
- Large aggregations might experience performance degradation.