Accelerating Fresh Data Exploration with Fluid ETL Pipelines
Maxwell Norfolk, Dong Xie · 2026
Read Accelerating Fresh Data Exploration with Fluid ETL Pipelines by Maxwell Norfolk, Dong Xie (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-22220.txt.
Description
Norfolk and Xie propose fluid ETL pipelines for exploring data while it is still arriving. Analysts often need to scan fresh data for main traits or odd values. Classic pipelines prep data with many routines, like parsing and indexing, to cover unknown goals. That prep work is costly. Fluid pipelines can start and stop prep routines on demand, without blocking ingestion. They run extra routines when idle or cheap spot capacity is free. Tests on a real dataset show the idea works.
Catalog record
| CREATOR | Maxwell Norfolk, Dong Xie |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 35584 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2603.22220