MMGraphRAG: Bridging Vision and Language with Interpretable Multimodal Knowledge Graphs
Xueyao Wan, Hang Yu · 2025
Read MMGraphRAG: Bridging Vision and Language with Interpretable Multimodal Knowledge Graphs by Xueyao Wan, Hang Yu (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2507-20804.txt.
Description
Xueyao Wan and Hang Yu build knowledge graphs that hold pictures and words together, in a 2025 paper. Language models invent facts because their knowledge is frozen at training time. Bolting on a knowledge graph helps, but most graphs store text only, leaving images out of the reasoning.
Their framework, MMGraphRAG, turns pictures into scene graphs — lists of objects and their relations — and merges them with text graphs. A linking method using spectral clustering matches entities across the two kinds by both meaning and graph position. The merged graph keeps its reasoning paths readable, so users can audit how an answer was reached. Grounded, inspectable answers matter in fields like medicine and law.
Catalog record
| CREATOR | Xueyao Wan, Hang Yu |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 81 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2507-20804