Evolutionary Prompt Optimization Discovers Emergent Multimodal Reasoning Strategies in Vision-Language Models
Sid Bharthulwar, John Rho, Katrina Brown · 2025
Read Evolutionary Prompt Optimization Discovers Emergent Multimodal Reasoning Strategies in Vision-Language Models by Sid Bharthulwar, John Rho, Katrina Brown (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2503-23503.txt.
Description
Sid Bharthulwar, John Rho, and Katrina Brown present a 2025 paper on evolving better prompts for vision language models. A prompt can be tuned like a trait under selection, with the best variants surviving each round. That style of search may find tricks no human would write by hand.
Their framework runs prompt variants through rounds of testing on visual tasks, keeping the winners. Across generations, the model struck on fresh tactics on its own. One tactic was to call a code tool to crop or recolor an image before answering. The authors show that naming this tool use in the prompt lifts scores a lot.
Catalog record
| CREATOR | Sid Bharthulwar, John Rho, Katrina Brown |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 42 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2503-23503