Overcoming Vision Language Model Challenges in Diagram Understanding: A Proof-of-Concept with XML-Driven Large Language Models Solutions
Shue Shiinoki, Ryo Koshihara, Hayato Motegi, Masumi Morishige · 2025
Read Overcoming Vision Language Model Challenges in Diagram Understanding: A Proof-of-Concept with XML-Driven Large Language Models Solutions by Shue Shiinoki, Ryo Koshihara, Hayato Motegi, Masumi Morishige (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2502-04389.txt.
Description
Shue Shiinoki, Ryo Koshihara, Hayato Motegi, and Masumi Morishige propose a text-driven workaround for diagram understanding in business documents. Vision-language models still struggle to extract structures and relationships from diagrams reliably. Office files, however, store diagram shapes, lines, and annotations as editable textual metadata.
The team extracted shape data from spreadsheet-based system design documents and fed it as text to a large language model. Compared with a vision-based method, the text-driven framework answered detailed structural questions more accurately. The approach extends to presentation and word-processing files with similar source data. The authors argue it can improve workflow efficiency and information analysis in real business settings.
Catalog record
| CREATOR | Shue Shiinoki, Ryo Koshihara, Hayato Motegi, Masumi Morishige |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 28 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2502-04389