Submodular Evaluation Subset Selection in Automatic Prompt Optimization
Jinming Nian, Zhiyuan Peng, Hongwei Shang, Dae Hoon Park, Yi Fang · 2026
Read Submodular Evaluation Subset Selection in Automatic Prompt Optimization by Jinming Nian, Zhiyuan Peng, Hongwei Shang, Dae Hoon Park, Yi Fang (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-03493.txt.
Description
Jinming Nian and four co-authors study a quiet choice inside automatic prompt optimization: which test examples should score each candidate prompt? Optimizers usually sample a small evaluation set at random and treat that choice as a detail. The authors argue the choice shapes the feedback signal, and therefore the final prompt, more than people assume.
Their method, SESS, picks evaluation subsets using submodular selection, a math framework for choosing items with diminishing added value. They show the goal is monotone and submodular under mild conditions, so a greedy algorithm carries guarantees. On GSM8K, MATH, and GPQA-Diamond benchmarks, chosen subsets produced better optimized prompts than random or rule-based picks. Careful test-set design can improve AI tuning without any bigger model.
Catalog record
| CREATOR | Jinming Nian, Zhiyuan Peng, Hongwei Shang, Dae Hoon Park, Yi Fang |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 17 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2601-03493