CLIP-Llama: A New Approach for Scene Text Recognition with a Pre-Trained Vision-Language Model and a Pre-Trained Language Model

Xiaoqing Zhao, Miaomiao Xu, Wushour Silamu, Yanbing Li · 2024

Read CLIP-Llama: A New Approach for Scene Text Recognition with a Pre-Trained Vision-Language Model and a Pre-Trained Language Model by Xiaoqing Zhao, Miaomiao Xu, Wushour Silamu, Yanbing Li (2024) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/pmc-11598184.txt.

Description

Xiaoqing Zhao, Miaomiao Xu, Wushour Silamu, and Yanbing Li present CLIP-Llama in a 2024 paper on reading text in wild photos. Scene text can run slanted, curved, blurred, or blocked, which breaks plain readers. Their build pairs a vision-language model with a big language model for its word sense.

Two decode paths run side by side: one reads the image traits, the other blends image and text sense to fix slips. The language model joins the second path as a corrector. On eleven bench sets the pair hit top marks, the team reports.

Catalog record

CREATORXiaoqing Zhao, Miaomiao Xu, Wushour Silamu, Yanbing Li
YEAR2024
COLLECTIONPAPERS
FORMATTXT
FILE SIZE26 KB
VERSION1.0
LICENSECC BY 4.0
RIGHTSCC BY
SOURCEEurope PMC

Source: https://europepmc.org/articles/PMC11598184

Browse the full library · About the Tower of Babel