Flaky Tests in a Large Industrial Database Management System: An Empirical Study of Fixed Issue Reports for SAP HANA Conference: 3rd International Flaky Tests Workshop 2026 ; April 12–18, 2026; Rio de Janeiro, Brazil 3rd International Flaky Tests Workshop 2026 (FTW ’26), April 12–18, 2026, Rio de Ja

Alexander Berndt, Thomas Bach, Sebastian Baltes · 2026

Read Flaky Tests in a Large Industrial Database Management System: An Empirical Study of Fixed Issue Reports for SAP HANA Conference: 3rd International Flaky Tests Workshop 2026 ; April 12–18, 2026; Rio de Janeiro, Brazil 3rd International Flaky Tests Workshop 2026 (FTW ’26), April 12–18, 2026, Rio de Ja by Alexander Berndt, Thomas Bach, Sebastian Baltes (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-03556.txt.

Description

A flaky test passes one run and fails the next on identical code, poisoning trust in automated checks. Alexander Berndt and 2 co-authors study flakiness inside SAP HANA, a large industrial database system, in a 2026 paper. Hand-labeling hundreds of fixed flakiness reports would be slow, so they use language models as annotators instead.

Consistency checks within and across models make the automatic labels trustworthy enough to map the root causes. Concurrency problems dominate, appearing in about 23 percent of the 559 analyzed reports. Different test types show different flakiness profiles, the authors find. They urge future mitigation research to test whether fixes generalize across test types rather than assuming one cause fits all.

Catalog record

CREATORAlexander Berndt, Thomas Bach, Sebastian Baltes
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE30 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2602-03556

Browse the full library · About the Tower of Babel