MERGE: A Modal Equilibrium Relational Graph Framework for Multi-Modal Knowledge Graph Completion
Yuying Shang, Kun Fu, Zequn Zhang, Li Jin, Zinan Liu, Shensi Wang, Shuchao Li, Stefanos Kollias · 2024
Read MERGE: A Modal Equilibrium Relational Graph Framework for Multi-Modal Knowledge Graph Completion by Yuying Shang, Kun Fu, Zequn Zhang, Li Jin, Zinan Liu, Shensi Wang, Shuchao Li, Stefanos Kollias (2024) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/pmc-11644511.txt.
Description
Yuying Shang and co-authors present a framework for filling gaps in knowledge graphs that mix text, images, and structure, in a 2024 paper. Completing such graphs means guessing missing facts. A subtle problem is missing modalities: many entities lack a picture or a description, and that absence can poison learning if ignored.
Their system, MERGE, builds three modality-specific graph networks that pass messages from neighbors, letting an entity borrow the missing modality's signal from its neighborhood. A low-rank tensor fusion then aligns the modalities at both structural and semantic levels, keeping the fused features interpretable. At prediction time, an interpolation re-ranking step rebalances how much each modality counts, without breaking each one's meaning. Tests on four public datasets showed the approach effective and robust, the authors report.
Catalog record
| CREATOR | Yuying Shang, Kun Fu, Zequn Zhang, Li Jin, Zinan Liu, Shensi Wang, Shuchao Li, Stefanos Kollias |
|---|---|
| YEAR | 2024 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 87 KB |
| VERSION | 1.0 |
| LICENSE | CC BY 4.0 |
| RIGHTS | CC BY |
| SOURCE | Europe PMC |