A dual contrastive framework

Yuan Sun, Zhao Zhang, Jorge Ortiz · 2024

Read A dual contrastive framework by Yuan Sun, Zhao Zhang, Jorge Ortiz (2024) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2412-10348.txt.

Description

Yuan Sun, Zhao Zhang, and Jorge Ortiz present AlignCap, a frame that teaches vision-language models to caption small regions, not just whole images. Their paper from twenty twenty-four names two drags on region skill. Spatial sense is weak, and coarse pre-training blurs the latent match between encoder and decoder. Most setups freeze both ends and tune the middle, so the blur persists.

A latent refinement module sharpens the conditioned features first. A semantic space alignment module then lifts the joint representation quality. Contrastive learning is woven through both in a fresh way. A general object detection pass preps the data, adding region-level spatial cues. Wide tests show clear gains in region captioning across tasks. The authors frame fine latent alignment as the missing step for region talk.

Catalog record

CREATORYuan Sun, Zhao Zhang, Jorge Ortiz
YEAR2024
COLLECTIONPAPERS
FORMATTXT
FILE SIZE9 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2412-10348

Browse the full library · About the Tower of Babel