air command group

Important

This feature is in Public Preview.

The air command group within the Databricks CLI submits and manages GPU workloads, such as training and inference, with AI Runtime.

For workload configuration fields, see Workload YAML reference. To submit your first workload, see Databricks CLI quickstart for AI Runtime.

databricks air run

Submit a workload from a YAML configuration. The output includes the job run ID and job run URL.

databricks air run --file YAML_PATH [flags]

Arguments

None

Options

-f, --file string

    Path to the workload YAML configuration. This option is required.

--watch

    Stream node 0 logs until the run reaches a terminal state. Default: false.

--override stringArray

    Override a YAML field using FIELD_PATH=VALUE. Repeat this option to override multiple fields.

--dry-run

    Validate the configuration without submitting a workload. Default: false.

--idempotency-key string

    Return an existing run when the key was previously used. Overrides idempotency_token in the YAML. Maximum: 64 characters.

Global flags

Examples

The following example submits a workload from train.yaml:

databricks air run --file train.yaml

Pass a configuration path as a separate argument to -h to display workload configuration help:

databricks air run -h config
databricks air run -h config.compute
databricks air run -h config.compute.accelerator_type

databricks air get

Get details for a run.

databricks air get JOB_RUN_ID [flags]

Arguments

JOB_RUN_ID

    A job run ID returned by databricks air run.

Options

Global flags

databricks air list

List runs. By default, the command lists active runs submitted by the current user.

databricks air list [flags]

Arguments

None

Options

--limit int

    Maximum number of runs to show. Default: 20.

--all-status

    Include runs in terminal states. Default: false.

--all-users

    Include runs submitted by other users. Default: false.

--filter stringArray

    Filter runs using KEY=VALUE. Repeat this option to require multiple conditions. Supported keys are accelerator_type, experiment, num_accelerators, and user.

Global flags

databricks air logs

Stream or fetch logs for a run.

databricks air logs JOB_RUN_ID [flags]

Arguments

JOB_RUN_ID

    A job run ID returned by databricks air run.

Options

--node int

    Select a zero-based node index. Default: 0.

--tail int

    Number of existing log lines to print. Default: 10000.

--minutes int

    Fetch logs from the last number of minutes. Cannot be combined with --tail. Default: 0.

--retry int

    Select a retry attempt. 0 is the initial attempt and -1 is the latest. Default: -1.

--download-to string

    Download complete logs to a directory. Downloads all nodes unless --node is set. Cannot be combined with --tail or --minutes.

Global flags

databricks air cancel

Cancel one or more runs by job run ID, or cancel all active runs submitted by the current user.

databricks air cancel [JOB_RUN_ID...] [flags]

Arguments

JOB_RUN_ID

    One or more job run IDs. Cannot be combined with --all.

Options

--all

    Cancel all active runs submitted by the current user. Default: false.

-y, --yes

    Skip the confirmation prompt used by --all. Default: false.

Global flags

databricks air pools list

List provisioned GPU capacity available to the current workspace.

Use the provisioned capacity ID in compute.pool_id.

databricks air pools list [flags]

Arguments

None

Options

Global flags

databricks air pools get

Get details for provisioned GPU capacity.

databricks air pools get [POOL_ID] [flags]

Arguments

POOL_ID

    The provisioned capacity ID. The argument is optional only when the workspace has exactly one provisioned capacity.

Options

Global flags

databricks air images push

Push a container image to Artifact Registry.

For prerequisites and instructions for pushing an image, see Push images with the Databricks CLI.

databricks air images push [flags]

Arguments

None

Options

-s, --source string

    The source container image as a name with an optional tag (NAME[:TAG]) or a digest (NAME@DIGEST). If omitted in an interactive terminal, the command prompts for the value.

--catalog string

    The destination Unity Catalog catalog. If omitted in an interactive terminal, the command prompts for the value.

--schema string

    The destination Unity Catalog schema. If omitted in an interactive terminal, the command prompts for the value.

--artifact string

    The destination artifact as ARTIFACT[:TAG]. Defaults to the source image name and tag, or the latest tag when the source has no tag.

--pull

    Pull the source image from its registry before pushing instead of reusing an available local image. Default: false.

Global flags

databricks air convert-to-dabs

Convert a workload YAML configuration to a Databricks Asset Bundle. This local command writes databricks.yml and launch files under generated_artifacts/.

This command is an optional bridge to an advanced production workflow. You can continue to use databricks air run without converting the configuration.

For the bundle workflow, see Schedule GPU workloads and compose tasks.

databricks air convert-to-dabs YAML_PATH [flags]

Arguments

YAML_PATH

    The path to the workload YAML configuration.

Options

--output-dir string

    Directory where the command writes the bundle. The code source must be inside this directory. Default: the directory that contains the input YAML file.

--force

    Overwrite generated bundle files that already exist. Default: false.

Global flags

Global flags

--debug

  Whether to enable debug logging.

-h or --help

    Display help for the Databricks CLI or the related command group or the related command.

--log-file string

    A string representing the file to write output logs to. If this flag is not specified then the default is to write output logs to stderr.

--log-format format

    The log format type, text or json. The default value is text.

--log-level string

    A string representing the log format level. If not specified then the log format level is disabled.

-o, --output type

    The command output type, text or json. The default value is text.

-p, --profile string

    The name of the profile in the ~/.databrickscfg file to use to run the command. If this flag is not specified then if it exists, the profile named DEFAULT is used.

--progress-format format

    The format to display progress logs: default, append, inplace, or json

-t, --target string

    If applicable, the bundle target to use

Migrate from the legacy Python CLI

The Databricks CLI and the deprecated Python-based air CLI use different command names and flags. Update scripts using the following mappings:

Python CLI Databricks CLI
air run -f train.yaml databricks air run -f train.yaml
air run -f train.yaml --override A=X B=Y databricks air run -f train.yaml --override A=X --override B=Y
air get run <run-id> databricks air get <run-id>
air list runs databricks air list --all-status
air list runs --active databricks air list
air -h config databricks air run -h config
air logs <run-id> databricks air logs <run-id>
air cancel <run-id> databricks air cancel <run-id>

Additional resources

  • Last updated on