Insight Reduces Databricks Costs
by 28% with Intel® Granulate™

Insight Faces Rising Databricks Costs

Infrastructure: Azure Databricks


With business growth often comes increasing, hard-to-manage costs. For Insight, its Databricks consumption reached 10% growth a month, with its largest all-purpose cluster comprising approximately 220 average cores or 2,640 DBUs per day, and scaling up to 17 nodes.

The IT consulting firm was already leveraging several optimization solutions and best practices, including cluster rightsizing, spot instances, and configuration tuning, but the Databricks costs continued to grow, sparking the need to look for a way to contain cloud costs without disrupting the exciting, consistent expansion of business.

At an annual, internal event, a group of Insight teammates learned about Intel Granulate as a solution to optimize their cloud spend through application performance optimization. Soon after, Insight and Intel began an engagement to deploy a pilot of Intel Granulate on a Databricks cluster to see if it could help reduce spend.

Optimizing Databricks With Intel Granulate

Insight required the solution to be simple to deploy and necessitate minimal engineering efforts to maintain. They also needed the optimization to be effective and complementary to existing optimization efforts.

Insight decided to try out the Intel Granulate Databricks optimizations by applying them on one cluster to start, with plans to expand across other Azure workloads if successful.

Upon activation, Intel Granulate identified patterns of consumption and determined areas of inefficiency in cloud workloads. Enterprise-ready, a POC was developed in only one week, allowing the project to move forward at an accelerated pace, with no code changes needed.