ai_mask function

Applies to: check marked yes Databricks SQL check marked yes Databricks Runtime

Important

This functionality is in Public Preview and HIPAA compliant.

During the preview:

The ai_mask() function allows you to invoke a state-of-the-art AI model to mask specified entities in a given text using SQL. This function uses a chat model serving endpoint made available by Databricks Foundation Model APIs.

Requirements

Apache 2.0 license

The underlying models that might be used at this time are licensed under the Apache 2.0 License, Copyright © The Apache Software Foundation. Customers are responsible for ensuring compliance with applicable model licenses.

Databricks recommends reviewing these licenses to ensure compliance with any applicable terms. If models emerge in the future that perform better according to Databricks's internal benchmarks, Databricks might change the model (and the list of applicable licenses provided on this page).

  • This function is only available on workspaces in regions that support AI Functions optimized for batch inference.
  • This function is not available on Azure Databricks SQL Classic.
  • Databricks Runtime 15.4 LTS or above is required. Databricks Runtime 18.2 or above is recommended for the best performance and access to the latest features.
  • Check the Databricks SQL pricing page.

Syntax

ai_mask(content, labels)

Arguments

  • content: A STRING expression.
  • labels: An ARRAY<STRING> literal. Each element is a short, plain-language description of a type of information to mask, such as 'person' or 'email'.

Returns

A STRING where each detected entity that matches one of the labels is replaced with [MASKED]. The replacement text is the same for every label.

If content is NULL, the result is NULL.

Examples

The following examples mask different types of entities. Because an AI model detects the entities, results can vary.

> SELECT ai_mask(
    'John Doe lives in New York. His email is john.doe@example.com.',
    array('person', 'email')
  );
 "[MASKED] lives in New York. His email is [MASKED]."

> SELECT ai_mask(
    'Contact me at 555-1234 or visit us at 123 Main St.',
    array('phone', 'address')
  );
 "Contact me at [MASKED] or visit us at [MASKED]"

To mask government IDs, payment details, and dates of birth, pass a label for each type:

> SELECT ai_mask(
    'SSN 123-45-6789, card 4111 1111 1111 1111, born on March 3, 1985.',
    array('ssn', 'credit card number', 'date of birth')
  );
 "SSN [MASKED], card [MASKED], born on [MASKED]."

To mask network identifiers, such as IP addresses and URLs, run the following:

> SELECT ai_mask(
    'Login from 192.168.1.10 failed. See https://internal.example.com/logs for details.',
    array('ip address', 'url')
  );
 "Login from [MASKED] failed. See [MASKED] for details."

To mask every row of a table column, call ai_mask in a query:

> SELECT
    ticket_id,
    ai_mask(ticket_text, array('person', 'email', 'phone', 'account number')) AS masked_text
  FROM support_tickets;
  • Viimeksi päivitetty