Visual information extraction from documents via classification-guided large vision-language models
Huafu Li, Guo Chen, Jia Xia, Lei Wang, Wei Du, Yun Yao, Weijun Peng, Liming Li · 2026
Read Visual information extraction from documents via classification-guided large vision-language models by Huafu Li, Guo Chen, Jia Xia, Lei Wang, Wei Du, Yun Yao, Weijun Peng, Liming Li (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/pmc-13139452.txt.
Description
Pulling names, dates, and figures out of scanned business documents is slow work when layouts vary and pages are stained or stamped. Huafu Li and co-authors present a method that pairs document sorting with a large vision-language model. The model first classifies the document type. Then tailored prompts guide it to extract the right fields, using a few examples in context.
The team tests the method on a real bidding data set with 16 certificate types. With no task training, it beats a strong supervised system by about 18 points in F1 score. It also stays steady against seals, watermarks, and low contrast. Light extra training lifts the score further, to about 94 percent. The authors frame the split design as a form of conditional computing that cuts uncertainty. They release code and models for others to use.
Catalog record
| CREATOR | Huafu Li, Guo Chen, Jia Xia, Lei Wang, Wei Du, Yun Yao, Weijun Peng, Liming Li |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 78 KB |
| VERSION | 1.0 |
| LICENSE | CC BY 4.0 |
| RIGHTS | CC BY |
| SOURCE | Europe PMC |