Grad-ECLIP: Gradient-based Visual and Textual Explanations for CLIP
Chenyang Zhao, Kun Wang, Janet H. Hsiao, Antoni B. Chan · 2025
Read Grad-ECLIP: Gradient-based Visual and Textual Explanations for CLIP by Chenyang Zhao, Kun Wang, Janet H. Hsiao, Antoni B. Chan (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2502-18816.txt.
Description
Chenyang Zhao and three co-authors build a tool to explain the choices of a leading image-text model in a 2025 paper. The model, CLIP, matches pictures with captions, but users cannot see which parts of a picture or words in a caption drove a match. Past explanation tools for such models lean on attention maps that are too sparse here to help.
Their method works backward from the match score through the model’s layers. It splits the score’s link to inner features and applies channel and space weights, giving clean heat maps over both the image and the text. The maps show which regions and which words pushed the match up or down. Tests, both by eye and by number, rate the maps as faithful guides to the model’s logic.
Catalog record
| CREATOR | Chenyang Zhao, Kun Wang, Janet H. Hsiao, Antoni B. Chan |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 86 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2502-18816