lakewatch command group

Note

Databricks CLI use is subject to the Databricks License and Databricks Privacy Notice, including any Usage Data provisions.

Important

Lakewatch is in Beta. Workspace admins can control access to this feature from the Previews page. See Manage workspace-level previews.

The lakewatch command group within the Databricks CLI contains commands to manage Lakewatch datasources, parsers, detection rules, and per-catalog configuration. See Lakewatch documentation.

The lakewatch command group requires Databricks CLI version 1.25.0 or above. The parser commands require version 1.27.0 or above. To check your version, run databricks -v.

databricks lakewatch create-datasource

Creates a new datasource.

databricks lakewatch create-datasource CATALOG_NAME SCHEMA_NAME DATASOURCE_NAME SCHEDULE [flags]

Arguments

CATALOG_NAME

    Name of the Unity Catalog catalog that contains the datasource. Required on create and cannot be changed afterward. Reads return the value as supplied; the catalog's current name is reported in effective_catalog_name.

SCHEMA_NAME

    Name of the Unity Catalog schema that contains the datasource. Required on create and cannot be changed afterward. Reads return the value as supplied; the schema's current name is reported in effective_schema_name.

DATASOURCE_NAME

    Name of the datasource in Unity Catalog, unique within its schema. Required on create and can be updated within the same catalog and schema. Get synchronizes this field with the current Unity Catalog name, also reported in effective_datasource_name.

SCHEDULE

    Schedule on which the datasource's ingestion job runs. A datasource cannot be paused, so a request that sets schedule.enabled to false is rejected. Continuous schedules are not supported for Unity Catalog connection datasources.

Options

--compute-mode DatasourceComputeMode

    Compute mode for the datasource's ingestion job. Supported values: DS_COMPUTE_PERFORMANCE_OPTIMIZED, DS_COMPUTE_STANDARD.

--description string

    Free-form description of the datasource.

--json JSON

    Either inline JSON string or @path/to/file.json with request body.

--name string

    Resource name of the datasource, in the format datasources/{datasource_id}.

--run-as-principal string

    The user or service principal that the datasource runs as, given as a user email or service principal application ID.

--warehouse-id string

    ID of the SQL warehouse that runs the checks Lakewatch performs to validate a create or update request, such as source and destination access checks.

Global flags

databricks lakewatch create-parser

Creates a parser and its first version in one call.

databricks lakewatch create-parser CATALOG_NAME SCHEMA_NAME PARSER_NAME SPEC [flags]

Arguments

CATALOG_NAME

    Name of the Unity Catalog catalog that contains the parser. Set on create. Reads return the value as supplied; the catalog's current name is reported in effective_catalog_name.

SCHEMA_NAME

    Name of the Unity Catalog schema that contains the parser. Set on create. Reads return the value as supplied; the schema's current name is reported in effective_schema_name.

PARSER_NAME

    Name of the parser, stored as supplied and fixed at create time. Must be a valid Unity Catalog identifier. A custom parser is not a Unity Catalog securable, and it cannot be renamed.

SPEC

    Spec of version 1, as a YAML document. Only create-parser accepts this argument. To read a version's spec, use get-parser-version. See Parser reference.

Options

--description string

    Description of version 1.

--json JSON

    Either inline JSON string or @path/to/file.json with request body.

--name string

    Resource name of the parser, in the format parsers/{parser_id}.

Global flags

Examples

The following example creates a parser named my_parser in the default schema of the main catalog, and returns the result as JSON:

databricks lakewatch create-parser main default my_parser 'fields:
  - name: source_ip
    type: STRING
    path: src_ip
time:
  coalesce:
    - path: ts
' -o json

databricks lakewatch create-parser-version

Appends a new immutable version to a parser and advances the parser's latest-version pointer. To edit a parser, create a new version. You can't change an existing version in place.

databricks lakewatch create-parser-version PARENT SPEC [flags]

Arguments

PARENT

    Resource name of the parser to add a version to, in the format parsers/{parser_id}.

SPEC

    The field definitions for this log source, as a YAML document. The spec is validated on write, and an invalid spec is rejected. See Parser reference.

Options

--description string

    Free-form description of this version, set when the version is created.

--json JSON

    Either inline JSON string or @path/to/file.json with request body.

--name string

    Resource name of the version.

Global flags

databricks lakewatch create-rule

Creates a new rule.

databricks lakewatch create-rule CATALOG_NAME SCHEMA_NAME RULE_NAME SEVERITY FIDELITY SCHEDULE OUTPUT [flags]

Arguments

CATALOG_NAME

    Name of the Unity Catalog catalog that contains the rule. Required. Cannot be changed after creation. Reads return the value as supplied; the catalog's current name is reported in effective_catalog_name.

SCHEMA_NAME

    Name of the Unity Catalog schema that contains the rule. Required. Cannot be changed after creation. Reads return the value as supplied; the schema's current name is reported in effective_schema_name.

RULE_NAME

    Name of the rule within its Unity Catalog schema. Required on create; can be updated within the same catalog and schema. Get synchronizes this field with the current Unity Catalog name, also reported in effective_rule_name.

SEVERITY

    Free-form severity label, such as Informational, Medium, High, or Critical.

FIDELITY

    Signal quality for this rule. Supported values: [FIDELITY_DEVELOPMENT, FIDELITY_HIGH, FIDELITY_INVESTIGATIVE, FIDELITY_MEDIUM].

SCHEDULE

    When the rule executes.

OUTPUT

    What to record in each signal.

Options

--category string

    Free-form rule category.

--content-resource-id string

    Immutable UUIDv4 identifying the authored content resource installed as this rule.

--deduplication RuleDeduplication

    Signal deduplication mode. Supported values: DEDUPLICATION_DEFAULT, DEDUPLICATION_DISABLED, DEDUPLICATION_ENABLED.

--deduplication-time-window string

    Time window for deduplication, such as 1h, 24h, 7d, or automatic.

--description string

    Free-form description.

--json JSON

    Either inline JSON string or @path/to/file.json with request body.

--name string

    Resource name, in the form rules/{rule_id}.

--objective string

    Long-form description of what this rule detects.

--run-as-principal string

    The principal whose identity this rule's job runs as, given as a user email or service-principal application ID.

--runbook string

    Long-form operational guidance carried from the authored rule metadata.

--suppress-until-time string

    If set, this rule's signals do not open new alerts until the given time.

Global flags

databricks lakewatch delete-datasource

Deletes a datasource by its resource name.

databricks lakewatch delete-datasource NAME [flags]

Arguments

NAME

    Resource name of the datasource, in the format datasources/{datasource_id}.

Global flags

databricks lakewatch delete-parser

Deletes a parser and all of its versions. The request is refused while any datasource references the parser.

databricks lakewatch delete-parser NAME [flags]

Arguments

NAME

    Resource name of the parser, in the format parsers/{parser_id}.

Global flags

databricks lakewatch delete-parser-version

Deletes one version of a parser. Returns NOT_FOUND when the parser or the version does not exist, and a conflict (HTTP 409) only while a datasource references the version. Deleting the last version leaves the parser in place with no versions.

databricks lakewatch delete-parser-version NAME [flags]

Arguments

NAME

    Resource name of the version, in the format parsers/{parser_id}/versions/{version}, where {version} is a version number written without a sign or leading zeros, such as 3. Unlike get-parser-version, this command rejects the alias latest and forms such as 007 or +3, so each delete targets one fixed version.

Global flags

databricks lakewatch delete-rule

Deletes a rule by its resource name.

databricks lakewatch delete-rule NAME [flags]

Arguments

NAME

    Resource name of the rule to delete. Format: rules/{rule_id}.

Global flags

databricks lakewatch get-alert-config

Returns the alert delivery configuration for a catalog.

databricks lakewatch get-alert-config NAME [flags]

Arguments

NAME

    Resource name of the config, in the format catalogs/{catalog}/alertConfig.

Global flags

databricks lakewatch get-builtin-parser

Returns a built-in parser's identity and latest-version pointer. Does not return the spec. To read a spec, use get-builtin-parser-version.

databricks lakewatch get-builtin-parser NAME [flags]

Arguments

NAME

    Resource name of the built-in parser, in the format builtin-parsers/{parser_id}.

Global flags

databricks lakewatch get-builtin-parser-version

Returns one version of a built-in parser, including its spec.

databricks lakewatch get-builtin-parser-version NAME [flags]

Arguments

NAME

    Resource name of the version, in the format builtin-parsers/{parser_id}/versions/{version}, where {version} is either a version number or the alias latest.

Global flags

databricks lakewatch get-common-config

Returns the common Lakewatch configuration for a catalog.

databricks lakewatch get-common-config NAME [flags]

Arguments

NAME

    Resource name of the config, in the format catalogs/{catalog}/commonConfig.

Global flags

databricks lakewatch get-datasource

Returns a single datasource by its resource name.

databricks lakewatch get-datasource NAME [flags]

Arguments

NAME

    Resource name of the datasource, in the format datasources/{datasource_id}.

Global flags

databricks lakewatch get-datasource-config

Returns the datasource ingestion configuration for a catalog.

databricks lakewatch get-datasource-config NAME [flags]

Arguments

NAME

    Resource name of the config, in the format catalogs/{catalog}/datasourceConfig.

Global flags

databricks lakewatch get-parser

Returns a parser's identity and latest-version pointer. Does not return the spec. To read a spec, use get-parser-version.

databricks lakewatch get-parser NAME [flags]

Arguments

NAME

    Resource name of the parser, in the format parsers/{parser_id}.

Global flags

databricks lakewatch get-parser-version

Returns one version of a parser, including its spec.

databricks lakewatch get-parser-version NAME [flags]

Arguments

NAME

    Resource name of the version, in the format parsers/{parser_id}/versions/{version}, where {version} is either a version number or the alias latest. latest returns NOT_FOUND when every version of the parser has been deleted.

Global flags

databricks lakewatch get-rule

Returns a single rule by its resource name.

databricks lakewatch get-rule NAME [flags]

Arguments

NAME

    Full resource name of the detection rule, in the form rules/{rule_id}.

Global flags

databricks lakewatch get-rule-config

Returns the detection-rule configuration for a catalog.

databricks lakewatch get-rule-config NAME [flags]

Arguments

NAME

    Resource name of the config, in the format catalogs/{catalog}/ruleConfig.

Global flags

databricks lakewatch list-builtin-parser-versions

Lists a built-in parser's published versions, oldest first, without their specs.

databricks lakewatch list-builtin-parser-versions PARENT [flags]

Arguments

PARENT

    Resource name of the built-in parser whose versions are listed, in the format builtin-parsers/{parser_id}.

Options

--limit int

    Maximum number of results to return.

--page-size int

    Maximum number of results to return in one page.

Global flags

databricks lakewatch list-builtin-parsers

Lists the built-in parsers that Databricks publishes, without their specs.

databricks lakewatch list-builtin-parsers [flags]

Options

--filter string

    Filter expression that restricts the results by tag. Supports a single clause, either tags.<key> = "<value>" to match a tag value or tags.<key>:* to match any parser that has the tag. Quote a key that contains a dot, for example tags."lw.vendor" = "AWS".

--limit int

    Maximum number of results to return.

--page-size int

    Maximum number of results to return in one page.

Global flags

databricks lakewatch list-datasources

Lists the datasources in a Unity Catalog catalog, optionally narrowed to one schema or filtered by name. Results are paginated. Datasources that are still being created or deleted are not listed.

databricks lakewatch list-datasources CATALOG_NAME [flags]

Arguments

CATALOG_NAME

    Name of the Unity Catalog catalog to list datasources from. Required; each request lists the datasources of a single catalog.

Options

--filter string

    Filter expression that restricts the results to datasources with a given name, for example datasource_name = "my_source".

--limit int

    Maximum number of results to return.

--page-size int

    Maximum number of results to return in one page.

--schema-name string

    Name of a Unity Catalog schema in catalog_name.

--view DatasourceView

    Which fields to return for each datasource. Supported values: DATASOURCE_VIEW_BASIC, DATASOURCE_VIEW_FULL.

Global flags

databricks lakewatch list-parser-versions

Lists a parser's versions, oldest first, without their specs.

databricks lakewatch list-parser-versions PARENT [flags]

Arguments

PARENT

    Resource name of the parser whose versions are listed, in the format parsers/{parser_id}.

Options

--limit int

    Maximum number of results to return.

--page-size int

    Maximum number of results to return in one page.

Global flags

databricks lakewatch list-parsers

Lists the parsers in a Unity Catalog catalog, without their specs.

databricks lakewatch list-parsers CATALOG_NAME [flags]

Arguments

CATALOG_NAME

    Name of the Unity Catalog catalog to list parsers from. Required; each request lists the parsers of a single catalog.

Options

--filter string

    Filter expression that restricts the results to parsers with a given name, for example parser_name = "my_parser".

--limit int

    Maximum number of results to return.

--page-size int

    Maximum number of results to return in one page.

--schema-name string

    Name of a Unity Catalog schema in catalog_name.

Global flags

databricks lakewatch list-rules

Lists rules, with optional filtering and pagination.

databricks lakewatch list-rules CATALOG_NAME [flags]

Arguments

CATALOG_NAME

    Name of the Unity Catalog catalog to list rules from. Required.

Options

--filter string

    Filter expression that restricts the results, such as rule_name = "my_rule" to return only the rule with that name.

--limit int

    Maximum number of results to return.

--page-size int

    Maximum number of rules to return.

--schema-name string

    Name of a Unity Catalog schema in catalog_name.

--view RuleView

    How much of each rule to return. Supported values: RULE_VIEW_BASIC, RULE_VIEW_FULL.

Global flags

databricks lakewatch update-alert-config

Updates the alert delivery configuration for a catalog.

databricks lakewatch update-alert-config NAME UPDATE_MASK [flags]

Arguments

NAME

    Resource name of the config, in the format catalogs/{catalog}/alertConfig.

UPDATE_MASK

    The field mask must be a single string, with multiple fields separated by commas (no spaces). The field path is relative to the resource object, using a dot (.) to navigate sub-fields (e.g., author.given_name). Specification of elements in sequence or map fields is not allowed, as only the entire collection field can be specified. Field names must exactly match the resource field names.

    A field mask of * indicates full replacement. It’s recommended to always explicitly list the fields being updated and avoid using * wildcards, as it can lead to unintended results if the API changes in the future.

Options

--default-security-alert-collation-window string

    Window for collating Signals into security alerts.

--default-system-alert-collation-window string

    Window for collating SystemEvents into system alerts.

--etag string

    Optimistic-concurrency precondition: if set and it does not match the resource's current etag, the update is rejected as a conflict (HTTP 409).

--json JSON

    Either inline JSON string or @path/to/file.json with request body.

--name string

    Resource name of the config, in the format catalogs/{catalog}/alertConfig.

Global flags

databricks lakewatch update-common-config

Updates the common Lakewatch configuration for a catalog.

databricks lakewatch update-common-config NAME UPDATE_MASK [flags]

Arguments

NAME

    Resource name of the config, in the format catalogs/{catalog}/commonConfig.

UPDATE_MASK

    The field mask must be a single string, with multiple fields separated by commas (no spaces). The field path is relative to the resource object, using a dot (.) to navigate sub-fields (e.g., author.given_name). Specification of elements in sequence or map fields is not allowed, as only the entire collection field can be specified. Field names must exactly match the resource field names.

    A field mask of * indicates full replacement. It’s recommended to always explicitly list the fields being updated and avoid using * wildcards, as it can lead to unintended results if the API changes in the future.

Options

--etag string

    Optimistic-concurrency precondition: if set and it does not match the resource's current etag, the update is rejected as a conflict (HTTP 409).

--json JSON

    Either inline JSON string or @path/to/file.json with request body.

--name string

    Resource name of the config, in the format catalogs/{catalog}/commonConfig.

--per-resource-permissions-enabled

    Whether users may grant permissions on individual resources rather than only at the Folder level.

Global flags

databricks lakewatch update-datasource

Updates an existing datasource.

databricks lakewatch update-datasource NAME UPDATE_MASK CATALOG_NAME SCHEMA_NAME DATASOURCE_NAME SCHEDULE [flags]

Arguments

NAME

    Resource name of the datasource, in the format datasources/{datasource_id}. Unlike the datasource's Unity Catalog location (catalog_name, schema_name, and datasource_name), the name does not change when the datasource or its catalog or schema is renamed in Unity Catalog.

UPDATE_MASK

    The field mask must be a single string, with multiple fields separated by commas (no spaces). The field path is relative to the resource object, using a dot (.) to navigate sub-fields (e.g., author.given_name). Specification of elements in sequence or map fields is not allowed, as only the entire collection field can be specified. Field names must exactly match the resource field names.

    A field mask of * indicates full replacement. It’s recommended to always explicitly list the fields being updated and avoid using * wildcards, as it can lead to unintended results if the API changes in the future.

CATALOG_NAME

    Name of the Unity Catalog catalog that contains the datasource. Required on create and cannot be changed afterward. Reads return the value as supplied; the catalog's current name is reported in effective_catalog_name.

SCHEMA_NAME

    Name of the Unity Catalog schema that contains the datasource. Required on create and cannot be changed afterward. Reads return the value as supplied; the schema's current name is reported in effective_schema_name.

DATASOURCE_NAME

    Name of the datasource in Unity Catalog, unique within its schema. Required on create and can be updated within the same catalog and schema. Get synchronizes this field with the current Unity Catalog name, also reported in effective_datasource_name.

SCHEDULE

    Schedule on which the datasource's ingestion job runs. A datasource cannot be paused, so a request that sets schedule.enabled to false is rejected. Continuous schedules are not supported for Unity Catalog connection datasources.

Options

--compute-mode DatasourceComputeMode

    Compute mode for the datasource's ingestion job. Supported values: DS_COMPUTE_PERFORMANCE_OPTIMIZED, DS_COMPUTE_STANDARD.

--description string

    Free-form description of the datasource.

--etag string

    Optimistic-concurrency precondition: if set to a non-empty value that does not match the resource's current etag, the update is rejected as a conflict (HTTP 409).

--json JSON

    Either inline JSON string or @path/to/file.json with request body.

--name string

    Resource name of the datasource, in the format datasources/{datasource_id}.

--run-as-principal string

    The user or service principal that the datasource runs as, given as a user email or service principal application ID.

--warehouse-id string

    ID of the SQL warehouse that runs the checks Lakewatch performs to validate a create or update request, such as source and destination access checks.

Global flags

databricks lakewatch update-datasource-config

Updates the datasource ingestion configuration for a catalog.

databricks lakewatch update-datasource-config NAME UPDATE_MASK [flags]

Arguments

NAME

    Resource name of the config, in the format catalogs/{catalog}/datasourceConfig.

UPDATE_MASK

    The field mask must be a single string, with multiple fields separated by commas (no spaces). The field path is relative to the resource object, using a dot (.) to navigate sub-fields (e.g., author.given_name). Specification of elements in sequence or map fields is not allowed, as only the entire collection field can be specified. Field names must exactly match the resource field names.

    A field mask of * indicates full replacement. It’s recommended to always explicitly list the fields being updated and avoid using * wildcards, as it can lead to unintended results if the API changes in the future.

Options

--default-compute-mode DatasourceComputeMode

    Default compute mode applied to datasource ingestion jobs when the individual datasource doesn't override it. Supported values: DS_COMPUTE_PERFORMANCE_OPTIMIZED, DS_COMPUTE_STANDARD.

--default-folder string

    Schema name of the registered folder (in this catalog) that a new datasource is placed in when the request does not specify a schema.

--etag string

    Optimistic-concurrency precondition: if set and it does not match the resource's current etag, the update is rejected as a conflict (HTTP 409).

--json JSON

    Either inline JSON string or @path/to/file.json with request body.

--name string

    Resource name of the config, in the format catalogs/{catalog}/datasourceConfig.

Global flags

databricks lakewatch update-rule

Updates an existing rule.

databricks lakewatch update-rule NAME UPDATE_MASK CATALOG_NAME SCHEMA_NAME RULE_NAME SEVERITY FIDELITY SCHEDULE OUTPUT [flags]

Arguments

NAME

    Resource name, in the form rules/{rule_id}. Assigned by the server on create; required on update.

UPDATE_MASK

    The set of fields to update. Only fields named in the mask are modified.

CATALOG_NAME

    Name of the Unity Catalog catalog that contains the rule. Required. Cannot be changed after creation. Reads return the value as supplied; the catalog's current name is reported in effective_catalog_name.

SCHEMA_NAME

    Name of the Unity Catalog schema that contains the rule. Required. Cannot be changed after creation. Reads return the value as supplied; the schema's current name is reported in effective_schema_name.

RULE_NAME

    Name of the rule within its Unity Catalog schema. Required on create; can be updated within the same catalog and schema. Get synchronizes this field with the current Unity Catalog name, also reported in effective_rule_name.

SEVERITY

    Free-form severity label, such as Informational, Medium, High, or Critical.

FIDELITY

    Signal quality for this rule. Supported values: [FIDELITY_DEVELOPMENT, FIDELITY_HIGH, FIDELITY_INVESTIGATIVE, FIDELITY_MEDIUM].

SCHEDULE

    When the rule executes.

OUTPUT

    What to record in each signal.

Options

--category string

    Free-form rule category.

--content-resource-id string

    Immutable UUIDv4 identifying the authored content resource installed as this rule.

--deduplication RuleDeduplication

    Signal deduplication mode. Supported values: DEDUPLICATION_DEFAULT, DEDUPLICATION_DISABLED, DEDUPLICATION_ENABLED.

--deduplication-time-window string

    Time window for deduplication, such as 1h, 24h, 7d, or automatic.

--description string

    Free-form description.

--etag string

    Optimistic-concurrency token from the latest read.

--json JSON

    Either inline JSON string or @path/to/file.json with request body.

--name string

    Resource name, in the form rules/{rule_id}.

--objective string

    Long-form description of what this rule detects.

--run-as-principal string

    The principal whose identity this rule's job runs as, given as a user email or service-principal application ID.

--runbook string

    Long-form operational guidance carried from the authored rule metadata.

--suppress-until-time string

    If set, this rule's signals do not open new alerts until the given time.

Global flags

databricks lakewatch update-rule-config

Updates the detection-rule configuration for a catalog.

databricks lakewatch update-rule-config NAME UPDATE_MASK [flags]

Arguments

NAME

    Resource name of the config, in the format catalogs/{catalog}/ruleConfig.

UPDATE_MASK

    The field mask must be a single string, with multiple fields separated by commas (no spaces). The field path is relative to the resource object, using a dot (.) to navigate sub-fields (e.g., author.given_name). Specification of elements in sequence or map fields is not allowed, as only the entire collection field can be specified. Field names must exactly match the resource field names.

    A field mask of * indicates full replacement. It’s recommended to always explicitly list the fields being updated and avoid using * wildcards, as it can lead to unintended results if the API changes in the future.

Options

--default-folder string

    Schema name of the registered folder (in this catalog) that a new rule is placed in when the request does not specify a schema.

--default-signal-collation-window string

    Default duration for signal deduplication when a rule doesn't set its own, written as compact day, hour, minute, and second components, for example 5d or 1h. There is no default. An empty value clears it.

--etag string

    Optimistic-concurrency precondition: if set and it does not match the resource's current etag, the update is rejected as a conflict (HTTP 409).

--json JSON

    Either inline JSON string or @path/to/file.json with request body.

--name string

    Resource name of the config, in the format catalogs/{catalog}/ruleConfig.

Global flags

Global flags

--debug

  Whether to enable debug logging.

-h or --help

    Display help for the Databricks CLI or the related command group or the related command.

--log-file string

    A string representing the file to write output logs to. If this flag is not specified then the default is to write output logs to stderr.

--log-format format

    The log format type, text or json. The default value is text.

--log-level string

    A string representing the log format level. If not specified then the log format level is disabled.

-o, --output type

    The command output type, text or json. The default value is text.

-p, --profile string

    The name of the profile in the ~/.databrickscfg file to use to run the command. If this flag is not specified then if it exists, the profile named DEFAULT is used.

--progress-format format

    The format to display progress logs: default, append, inplace, or json

-t, --target string

    If applicable, the bundle target to use