Conformal Prediction for Zero-Shot Models
Julio Silva-RodríguezIsmail Ben AyedJose DolzÉTS Montréal · 2025
Read Conformal Prediction for Zero-Shot Models by Julio Silva-RodríguezIsmail Ben AyedJose DolzÉTS Montréal (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-24693.txt.
Description
Julio Silva-Rodríguez, Ismail Ben Ayed, and Jose Dolz study how to make the predictions of zero-shot vision-language models come with reliable guarantees. Models like CLIP adapt to new tasks without retraining, but their certainty is rarely checked. Conformal prediction gives such guarantees from a small labeled calibration set. A catch arises here: these models were trained once on a hidden source domain that differs from the target task, and that drift weakens the prediction sets.
The authors propose Conf-OT, a method that works across the combined calibration and query sets at once. It solves an optimal transport problem to bridge the domain gap, without extra data splits, while keeping the coverage guarantee. Tests span 15 datasets and three scoring rules. The method improves set efficiency by up to 20 percent and runs about 15 times faster than rival approaches.
Catalog record
| CREATOR | Julio Silva-RodríguezIsmail Ben AyedJose DolzÉTS Montréal |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 130 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2505-24693