Enhancing Vision-Language Compositional Understanding with Multimodal Synthetic Data
Haoxin Li, Boyang Li · 2025
Read Enhancing Vision-Language Compositional Understanding with Multimodal Synthetic Data by Haoxin Li, Boyang Li (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2503-01167.txt.
Description
Li and Li make synthetic training pairs that teach vision-language models to notice small wording swaps. A model that truly grasps composition should tell apart people holding surfboards from people holding shovels. Their 2025 method, SPARCL, forges such near-twin image and caption pairs with a fast image generator steered by features from a real photo.
A style transfer pass then keeps the forged image close to the real one outside the changed spot. A self-adjusting loss drops suspect pairs and leans on hard, informative ones. Across four composition tests, training with the forged data lifted a standard model's average score by more than eight points. It also beat prior methods on three of the four tests. The authors frame forgery as cheap teaching data.
Catalog record
| CREATOR | Haoxin Li, Boyang Li |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 51 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2503-01167