Efficient Vocabulary-Free Fine-Grained Visual Recognition in the Age of Multimodal LLMs

Hari Chandana Kuchibhotla, Sai Srinivas Kancheti, Abbavaram Gowtham Reddy, Vineeth N Balasubramanian · 2025

Read Efficient Vocabulary-Free Fine-Grained Visual Recognition in the Age of Multimodal LLMs by Hari Chandana Kuchibhotla, Sai Srinivas Kancheti, Abbavaram Gowtham Reddy, Vineeth N Balasubramanian (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-01064.txt.

Description

Telling apart near-identical categories, like bird species, is hard for image models. The task normally needs big labeled datasets from experts. Hari Chandana Kuchibhotla and co-authors remove the label list entirely. In their vocabulary-free setting, a model must name fine-grained classes it was never given. Big multimodal models can do this, but querying them per image is slow and costly. Their method, NeaR, uses such a model to label a small unlabeled set once. A CLIP model is then fine-tuned on those noisy labels.

The approach is built to absorb noise and open-ended answers. It sets a new benchmark for efficient fine-grained recognition without a vocabulary.

Catalog record

CREATORHari Chandana Kuchibhotla, Sai Srinivas Kancheti, Abbavaram Gowtham Reddy, Vineeth N Balasubramanian
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE71 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2505-01064

Browse the full library · About the Tower of Babel