Introduction

Every Azure Databricks workload runs on compute resources, but choosing the wrong compute type or configuration leads to unnecessary costs, poor performance, or blocked functionality. Serverless compute starts in seconds but doesn't support RDD APIs. Classic compute offers complete flexibility but requires more management overhead. SQL warehouses excel at analytical queries while job clusters optimize for automated workflows. Understanding these differences helps you match compute to workload requirements.

Beyond selecting a compute type, configuration decisions shape how your workload performs. Node types determine processing capacity and memory availability. Autoscaling balances cost and responsiveness. Access permissions control who can use compute resources while library installations provide the dependencies your code needs. Each configuration choice affects multiple dimensions—performance, cost, security, and operational complexity.

Getting compute configuration right matters throughout the development lifecycle. During exploration, you need fast iteration cycles and minimal setup overhead. In production, stability and cost efficiency take priority. Photon acceleration can double query performance for SQL workloads. Instance pools reduce startup time but require paying for idle capacity. Dedicated access modes enable secure group collaboration while enforcing permission boundaries.

In this module, you learn to evaluate compute options systematically and configure resources that match your workload characteristics. You discover when serverless compute provides the best experience, how to tune performance settings for different scenarios, and best practices for managing access and dependencies across your organization.