CAPO: Cost-Aware Prompt Optimization

Tom Zehle, Moritz Schlager, Timo Heiß, Matthias Feurer · 2025

Read CAPO: Cost-Aware Prompt Optimization by Tom Zehle, Moritz Schlager, Timo Heiß, Matthias Feurer (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2504-16005.txt.

Description

Tom Zehle, Moritz Schlager, Timo Heiß, and Matthias Feurer tackle the cost of automatic prompt optimization. Language model output depends heavily on prompt wording, but searching for the best prompt burns many model calls and tokens. Their algorithm, CAPO, borrows ideas from AutoML to cut that cost. It evolves prompts with language models as operators, uses racing to drop weak candidates early, and balances accuracy against prompt length.

CAPO tunes instructions and few-shot examples together and uses task descriptions for robustness. Across datasets and models it beat leading discrete methods in 11 of 15 cases, with accuracy gains up to 21 points. It reached strong results on smaller budgets and stayed robust to starting prompts.

Catalog record

CREATORTom Zehle, Moritz Schlager, Timo Heiß, Matthias Feurer
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE77 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2504-16005

Browse the full library · About the Tower of Babel