Predicting Lakehouse Performance in Clouds: An Empirical Exploration of Query Runtime Variance

James Nurdin, Wei Liu, Richard Mccreadie, Lauritz Thamsen · 2026

Read Predicting Lakehouse Performance in Clouds: An Empirical Exploration of Query Runtime Variance by James Nurdin, Wei Liu, Richard Mccreadie, Lauritz Thamsen (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2606-03464.txt.

Description

Nurdin, Liu, Mccreadie, and Thamsen measure how much lakehouse query times vary in the cloud. Lakehouse systems now run much analytics work on shared platforms. Predicting query time helps balance money, resource, and carbon costs. Yet the same query can run at very different speeds from run to run. That variance had not been studied in a systematic way for lakehouses.

The team ran Kubernetes setups on three public clouds and one private cloud. Multiple data scales and three benchmarks were covered. Repeated runs of one query differed by nearly twofold in time. A factor study probes causes like data locality, neighbor load, and caching. Prediction models suffer under this variance. Fixing key sources of it cuts prediction error by up to 80 percent. A low-carbon scheduling test shows real savings when variance is counted in. Operators get hard numbers for a cost they often ignore.

Catalog record

CREATORJames Nurdin, Wei Liu, Richard Mccreadie, Lauritz Thamsen
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE49 KB
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2606.03464

Browse the full library · About the Tower of Babel