CASPER — Change-Aware Slice Prioritization for Efficient Regression Testing of LLM-based systems
Biruk Asmare Muse, Lionel Briand, Yiwei Lu, Keheliya Gallaba · 2026
Read CASPER — Change-Aware Slice Prioritization for Efficient Regression Testing of LLM-based systems by Biruk Asmare Muse, Lionel Briand, Yiwei Lu, Keheliya Gallaba (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2608-00378.txt.
Introduction
Muse and co-authors present CASPER for regression testing of LLM-based systems. Single regression cases carry little signal about system-level regressions. One failure may not mark a meaningful regression. Whole-system scores are too coarse to name affected behaviours. Test suite slices offer the middle level they need.
CASPER finds slices of semantically related cases with consistent performance traits. An evolutionary method identifies those slices. Given a prompt- or model-level change, it ranks slices by regression likelihood. Behavioural facts from execution logs drive that ranking. The test bed is software issue resolution. CASPER made more consistent slices than clustering baselines. Regressed-slice ranking beat a random baseline across changes and budgets.
Catalog record
| CREATOR | Biruk Asmare Muse, Lionel Briand, Yiwei Lu, Keheliya Gallaba |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 140 KB |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2608.00378