Performance articles

Every BricksNotes article on Performance: 15 practical pieces on Databricks and data engineering, newest first.

Spill (Memory) and Spill (Disk) in Apache Spark: What They Actually Mean
Your Databricks Job Took 42 Minutes Instead of 8. What Changed?
Every Data Engineer Faces These 50 Problems. The Best Engineers Know the Solutions.
The checkpoint trap is finally fixable: resize stateful Spark streams without starting over
You partitioned the table. It got slower. Partitioning and liquid clustering, explained.
The small files problem in Spark and Databricks, explained simply
The table had 40,000 tiny files. The small file problem in Databricks, explained
Same result, 3 seconds or 30 minutes. Query optimization in Databricks
The join that ran for seven hours. Broadcast and shuffle joins in Databricks
One task ran for forty minutes. Data skew in Databricks, explained
All BricksNotes articles