Azure Daily 2022 by Gordon

Generally available: Azure Databricks SQL Pro

Beginning in November, Databricks is rolling out a new compute option called Databricks SQL Pro, joining the SQL product family of Classic and Serverless. Like Serverless SQL, SQL Pro includes performance and integration features that expand the SQL experience on the Lakehouse Platform. The primary difference is that SQL Pro keeps compute in the customer's account.

Azure Databricks SQL Pro’s features include:

Predictive I/O - Predictive I/O boosts query performance by leveraging the years of experience Databricks has in building large AI/ML systems to make the lakehouse a more intelligent data warehouse
- Applying deep learning techniques to our data lakehouse
- Determining the most efficient access pattern to read data
- For select queries, Predictive I/O calculates probabilities to anticipate where the next row match will occur, only reading that data from cloud storage
Native Geospatial Features - 30+ built-in H3 expressions for geospatial processing and analysis in Photon-enabled clusters, available in SQL, Scala, and Python
Query federation - Databricks Warehouse now supports the ability to query live data from various databases through federation capability. Query federation allows BI applications to integrate data in the lakehouse and external data sources, providing a rich query experience
Workflows integrations - Invoke DBSQL tasks like queries, alerts, and running dashboards from within Databricks Workflows

Many more feature and performance improvements are on the way, such as Materialized Views, and Python UDFs. SQL Pro is generally available everywhere Databricks SQL Classic is available.

Source: Generally available: Azure Databricks SQL Pro

Azure Databricks