Submit feedback on
Underuse of Serverless for Short or Interactive Workloads
We've received your feedback.
Thanks for reaching out!
Oops! Something went wrong while submitting the form.
Close
Underuse of Serverless for Short or Interactive Workloads
Nicole Boyd
Service Category
Compute
Cloud Provider
Databricks
Service Name
Databricks SQL
Inefficiency Type
Inefficient Configuration
Explanation

Many organizations continue running short-lived or low-intensity SQL workloads — such as dashboards, exploratory queries, and BI tool integrations — on traditional clusters. This leads to idle compute, overprovisioning, and high baseline costs, especially when the clusters are always-on. Databricks SQL Serverless is optimized for bursty, interactive use cases with auto-scaling and pay-per-second pricing, making it better suited for this class of workloads. Failing to migrate to serverless for these patterns results in unnecessary cost without performance benefit.

Relevant Billing Model

Serverless SQL compute is billed per second of execution time based on the size of the virtual warehouse used. Unlike full clusters, serverless compute does not incur idle time charges and automatically scales based on demand. Continuing to use traditional job or all-purpose clusters for low-throughput SQL workloads results in persistent infrastructure costs that serverless options are purpose-built to eliminate.

Detection
  • Identify SQL endpoints or clusters supporting dashboard or ad-hoc workloads with long idle periods between executions
  • Review query duration and frequency — flag endpoints that are infrequently used or support low-throughput workloads
  • Check whether teams are using dedicated clusters for interactive BI tools (e.g., Power BI, Tableau)
  • Look for cost spikes associated with idle or underutilized SQL clusters versus expected user interaction volume
Remediation
  • Migrate lightweight SQL workloads and dashboards to Databricks SQL Serverless
  • Enable serverless for high-concurrency, low-compute scenarios where persistent compute isn’t needed
  • Set policies or guidelines to default to serverless for interactive workloads unless specific performance reasons require otherwise
  • Educate data teams on the appropriate use cases for serverless versus provisioned clusters
Relevant Documentation
  • Databricks SQL Serverless Compute Overview
  • Databricks SQL Pricing
  • Configure SQL Warehouses
Submit Feedback