Cross-engine attribute-based access controls (ABAC)

External engines can read Unity Catalog tables with fine-grained access controls enforced. This includes both attribute-based access controls (ABAC) policies and standalone row filters and column masks, all dynamically enforced even when querying from outside Azure Databricks.

When an external engine queries a table with ABAC policies, row filters, or column masks attached, Azure Databricks uses a specialized serverless compute layer to filter and return sanitized data to the external engine.

Requirements

To enforce fine-grained access controls on tables queried from external engines, you must complete the following:

Create a managed Delta table with catalog commits

To create a new managed Delta table with catalog commits (requires Databricks Runtime 16.4 and above):

CREATE TABLE <catalog>.<schema>.<table> (id INT, name STRING)
USING delta
TBLPROPERTIES ('delta.feature.catalogManaged' = 'supported');

To upgrade an existing managed table (requires Databricks Runtime 18.0 and above):

ALTER TABLE <catalog>.<schema>.<table>
SET TBLPROPERTIES ('delta.feature.catalogManaged' = 'supported');

After you create the table, you can apply ABAC policies, row filters, or column masks.

See Create a policy or Manually apply row filters and column masks.

Read tables with Apache Spark (Delta)

Configure Apache Spark with Delta-Spark 4.1 or above and Unity Catalog Spark connector 0.4 or above.

"spark.sql.extensions": "io.delta.sql.DeltaSparkSessionExtension",
"spark.sql.catalog.spark_catalog": "org.apache.spark.sql.delta.catalog.DeltaCatalog",
"spark.sql.catalog.<uc-catalog-name>": "io.unitycatalog.spark.UCSingleCatalog",
"spark.sql.catalog.<uc-catalog-name>.uri": "<workspace-url>",
"spark.sql.catalog.<uc-catalog-name>.auth.type": "oauth",
"spark.sql.catalog.<uc-catalog-name>.auth.oauth.uri": "<oauth-token-endpoint>",
"spark.sql.catalog.<uc-catalog-name>.auth.oauth.clientId": "<oauth-client-id>",
"spark.sql.catalog.<uc-catalog-name>.auth.oauth.clientSecret": "<oauth-client-secret>",
"spark.sql.catalog.<uc-catalog-name>.serverSidePlanning.enabled": "true",
"spark.sql.defaultCatalog": "<uc-catalog-name>",
"spark.jars.packages": "io.delta:delta-spark_4.0_2.13:4.1.0,io.delta:delta-iceberg_2.13:4.1.0,io.unitycatalog:unitycatalog-spark_2.13:0.4.0,org.apache.hadoop:hadoop-azure:3.4.2"

Note

Set serverSidePlanning.enabled to true to enable fine-grained access control enforcement from external engines.

Substitute the following variables:

  • <uc-catalog-name>: The name of the catalog in Unity Catalog that contains your tables.
  • <workspace-url>: The Azure Databricks workspace URL, including the workspace ID.
  • <oauth-token-endpoint>: OAuth token endpoint URL. See Authorize service principal access to Azure Databricks with OAuth.
  • <oauth-client-id>: OAuth client ID for the authenticating principal.
  • <oauth-client-secret>: OAuth client secret for the authenticating principal.

Read tables with Apache Spark (Iceberg)

Configure Apache Spark with Iceberg-Spark 1.11 or above and Apache Spark 4.0 or above.

"spark.sql.extensions": "org.apache.iceberg.spark.extensions.IcebergSparkSessionExtensions",
"spark.sql.catalog.<uc-catalog-name>": "org.apache.iceberg.spark.SparkCatalog",
"spark.sql.catalog.<uc-catalog-name>.type": "rest",
"spark.sql.catalog.<uc-catalog-name>.uri": "<workspace-url>/api/2.1/unity-catalog/iceberg-rest/",
"spark.sql.catalog.<uc-catalog-name>.credential": "<oauth-client-id>:<oauth-client-secret>",
"spark.sql.catalog.<uc-catalog-name>.oauth2-server-uri": "<oauth-token-endpoint>",
"spark.sql.catalog.<uc-catalog-name>.warehouse": "<uc-catalog-name>",
"spark.sql.catalog.<uc-catalog-name>.cache-enabled": "false",
"spark.sql.defaultCatalog": "<uc-catalog-name>"

Substitute the following variables:

  • <uc-catalog-name>: The name of the catalog in Unity Catalog that contains your tables.
  • <workspace-url>: The Azure Databricks workspace URL, including the workspace ID.
  • <oauth-token-endpoint>: OAuth token endpoint URL. See Authorize service principal access to Azure Databricks with OAuth.
  • <oauth-client-id>: OAuth client ID for the authenticating principal.
  • <oauth-client-secret>: OAuth client secret for the authenticating principal.

Query data

You can query the table using Apache Spark SQL or DataFrame APIs. Azure Databricks enforces fine-grained access policies behind the scenes.

SELECT * FROM <uc-catalog-name>.<schema>.<table>;

Warning

Concurrent writes during query planning can cause the same table to be read from different table snapshots in self-join and multi-scan queries, potentially resulting in incorrect results.

Write to tables with ABAC policies

External engines can write to managed tables with ABAC policies when the authenticated principal is exempted from the policies.

  1. Add the principal, or an account group containing it, to the EXCEPT clause (Except for in Catalog Explorer) of every applicable ABAC row filter and column mask policy. Include policies inherited from the metastore, catalog, or schema. See Edit a policy and CREATE POLICY.
  2. Grant the principal USE CATALOG, USE SCHEMA, EXTERNAL USE SCHEMA, and the table privileges needed to read and write (SELECT and MODIFY).
  3. Configure a supported table for external writes from a Delta client or an Iceberg client.
  4. Authenticate the external client as the exempt principal.

For example, you run the following statements from an external Iceberg or Delta client to write to and read a managed table where the principal is exempt from all ABAC policies:

INSERT INTO <uc-catalog-name>.<schema>.<table> (id, name)
VALUES (1, 'Example customer');

SELECT * FROM <uc-catalog-name>.<schema>.<table>;

See Access Databricks data using external systems and Attribute-based access control in Unity Catalog.

Serverless compute costs

Cross-engine ABAC uses serverless compute resources to enforce fine-grained access policies server-side. Customers are charged for these resources. For pricing information, see Cross-engine ABAC pricing.

Users with access to the billing system table can query system.billing.usage to see how much they've been charged. For example, the following query breaks down compute costs by user:

SELECT usage_date,
sku_name,
 identity_metadata.run_as,
SUM(usage_quantity) AS `DBUs consumed by cross-engine ABAC`
FROM system.billing.usage
WHERE usage_date BETWEEN '2026-06-01' AND '2026-07-01'
 AND billing_origin_product = 'EXTERNAL_COMPATIBILITY'
GROUP BY 1, 2, 3 ORDER BY 1;

Limitations

  • Dynamic views are not supported.
  • Projecting VARIANT columns is not supported.
  • Filtering on BINARY columns is not supported.
  • Column-masking functions whose return type differs from the original column type are not supported.
  • Large aggregations might experience performance degradation.