Submodular Evaluation Subset Selection in Automatic Prompt Optimization

Jinming Nian, Zhiyuan Peng, Hongwei Shang, Dae Hoon Park, Yi Fang · 2026

Read Submodular Evaluation Subset Selection in Automatic Prompt Optimization by Jinming Nian, Zhiyuan Peng, Hongwei Shang, Dae Hoon Park, Yi Fang (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-03493.txt.

Description

Jinming Nian and four co-authors study a quiet choice inside automatic prompt optimization: which test examples should score each candidate prompt? Optimizers usually sample a small evaluation set at random and treat that choice as a detail. The authors argue the choice shapes the feedback signal, and therefore the final prompt, more than people assume.

Their method, SESS, picks evaluation subsets using submodular selection, a math framework for choosing items with diminishing added value. They show the goal is monotone and submodular under mild conditions, so a greedy algorithm carries guarantees. On GSM8K, MATH, and GPQA-Diamond benchmarks, chosen subsets produced better optimized prompts than random or rule-based picks. Careful test-set design can improve AI tuning without any bigger model.

Catalog record

CREATORJinming Nian, Zhiyuan Peng, Hongwei Shang, Dae Hoon Park, Yi Fang
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE17 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2601-03493

Browse the full library · About the Tower of Babel