A dual contrastive framework
Yuan Sun, Zhao Zhang, Jorge Ortiz · 2024
Read A dual contrastive framework by Yuan Sun, Zhao Zhang, Jorge Ortiz (2024) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2412-10348.txt.
Description
Yuan Sun, Zhao Zhang, and Jorge Ortiz present AlignCap, a frame that teaches vision-language models to caption small regions, not just whole images. Their paper from twenty twenty-four names two drags on region skill. Spatial sense is weak, and coarse pre-training blurs the latent match between encoder and decoder. Most setups freeze both ends and tune the middle, so the blur persists.
A latent refinement module sharpens the conditioned features first. A semantic space alignment module then lifts the joint representation quality. Contrastive learning is woven through both in a fresh way. A general object detection pass preps the data, adding region-level spatial cues. Wide tests show clear gains in region captioning across tasks. The authors frame fine latent alignment as the missing step for region talk.
Catalog record
| CREATOR | Yuan Sun, Zhao Zhang, Jorge Ortiz |
|---|---|
| YEAR | 2024 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 9 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2412-10348