Databricks control plane vs compute plane (data plane), explained

Databricks is a managed platform for running Apache Spark, SQL warehouses, notebooks, jobs and ML on data that sits in cloud object storage such as S3, ADLS or GCS. To understand its security model, networking and costs, you need one picture: Databricks is split into a control plane that Databricks runs, and a compute plane where your data is actually processed.

Older docs and many interview questions call the second one the data plane. Databricks now says compute plane, and there are two kinds of it: classic and serverless.

The one-paragraph answer

The control plane is the set of backend services Databricks manages in its own account: the web app, the REST APIs, the job scheduler, cluster management and workspace configuration. The compute plane is where clusters and SQL warehouses run and where your data is read and transformed. Classic compute runs in your cloud account, inside your virtual network. Serverless compute runs in Databricks’ account, in a per-workspace network boundary, in the same region as your workspace.

Side by side

	Control plane	Classic compute plane	Serverless compute plane
Runs in	Databricks’ cloud account	Your cloud account (your VPC / VNet)	Databricks’ cloud account
What lives there	Web UI, REST APIs, job scheduling, cluster management, workspace configuration	Cluster VMs for all-purpose compute, jobs and classic SQL warehouses	Serverless notebooks, jobs, pipelines and SQL warehouses
Processes your data	No	Yes	Yes
Who manages networking	Databricks	You (subnets, routes, firewalls, egress)	Databricks, with per-workspace isolation
Who pays for the VMs	Included in DBUs	You pay your cloud provider plus DBUs	Bundled into serverless DBU prices

What a request looks like

1. You open a notebook in the browser. The page is served by the control plane.
2. You run a cell. The control plane sends the command to a cluster.
3. The cluster, in the compute plane, reads your tables from cloud storage, runs the Spark job and sends results back for display.
4. A scheduled job works the same way: the control plane’s scheduler starts compute when the job is due; the compute does the work.

The control plane orchestrates; the compute plane does the heavy lifting. That separation is why Databricks can update the UI and APIs without touching your clusters, and why your raw data does not need to pass through the control plane to be processed.

Where your data lives

• Your data (Delta tables, files, Unity Catalog volumes) lives in cloud object storage. With classic workspaces that storage is in your cloud account; you grant Databricks access to it.
• Workspace storage is a separate bucket or storage account that classic workspaces use for workspace system data, legacy DBFS root and, in some setups, the default Unity Catalog catalog. Databricks’ docs are blunt about it: don’t modify or delete it, or the workspace can’t be recovered.
• Serverless workspaces use fully managed default storage instead, and you can still connect your own storage for your data.

Interview shortcut: “Data is processed in the compute plane; the control plane manages metadata and orchestration.”

How classic clusters connect: secure cluster connectivity

With secure cluster connectivity (also called no public IP), classic cluster nodes have no public IP addresses and your network has no open inbound ports. Each cluster opens an outbound HTTPS connection (port 443) to a relay in the control plane when it starts, and the control plane sends its cluster management commands back through that tunnel.

What this means in practice:

• Your firewall needs to allow outbound 443 to the Databricks relay endpoints for your region. Databricks recommends allowlisting their domain names, not IP addresses.
• Clusters still need egress for anything else they call (package mirrors, external APIs), usually through a NAT gateway.
• On Azure, new workspaces enable secure cluster connectivity by default, and Microsoft documents plans to require it for all classic workspaces.

Serverless compute doesn’t use this mechanism, but its compute also has no public IP addresses.

Classic or serverless?

Choose classic when you need full control of the network path (custom routing, inspection firewalls, private endpoints to on-prem systems), the full choice of instance types, or when security policy says compute must run in your account.

Choose serverless when you want compute to start in seconds, don’t want to manage VMs, autoscaling and idle clusters, or are running SQL warehouses and short jobs where startup time dominates.

Many teams use both: serverless SQL warehouses for analysts and dashboards, classic job clusters for heavy pipelines with special networking.

Common questions

Does my data leave my account? With classic compute, data is processed by VMs in your account. Results you display in a notebook are returned to your browser through the control plane, so treat displayed output like any other data leaving the cluster. With serverless, processing happens in Databricks’ account under its isolation controls, so check your compliance requirements before switching.

Where do I configure what? Network settings for classic compute (VPC/VNet injection, private endpoints) are set when you create the workspace; serverless network policies are configured at the account level.

For a deeper walkthrough with diagrams, read the free lesson Control plane & compute plane. The Databricks course then goes from this architecture through Unity Catalog, Delta Lake, jobs and pipelines to a full lakehouse project, and the first three lessons are free. If you’re new to Spark itself, start with the PySpark course.