Conformal Prediction for Zero-Shot Models

Julio Silva-RodríguezIsmail Ben AyedJose DolzÉTS Montréal · 2025

Read Conformal Prediction for Zero-Shot Models by Julio Silva-RodríguezIsmail Ben AyedJose DolzÉTS Montréal (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-24693.txt.

Description

Julio Silva-Rodríguez, Ismail Ben Ayed, and Jose Dolz study how to make the predictions of zero-shot vision-language models come with reliable guarantees. Models like CLIP adapt to new tasks without retraining, but their certainty is rarely checked. Conformal prediction gives such guarantees from a small labeled calibration set. A catch arises here: these models were trained once on a hidden source domain that differs from the target task, and that drift weakens the prediction sets.

The authors propose Conf-OT, a method that works across the combined calibration and query sets at once. It solves an optimal transport problem to bridge the domain gap, without extra data splits, while keeping the coverage guarantee. Tests span 15 datasets and three scoring rules. The method improves set efficiency by up to 20 percent and runs about 15 times faster than rival approaches.

Catalog record

CREATORJulio Silva-RodríguezIsmail Ben AyedJose DolzÉTS Montréal
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE130 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2505-24693

Browse the full library · About the Tower of Babel