MERGE: A Modal Equilibrium Relational Graph Framework for Multi-Modal Knowledge Graph Completion

Yuying Shang, Kun Fu, Zequn Zhang, Li Jin, Zinan Liu, Shensi Wang, Shuchao Li, Stefanos Kollias · 2024

Read MERGE: A Modal Equilibrium Relational Graph Framework for Multi-Modal Knowledge Graph Completion by Yuying Shang, Kun Fu, Zequn Zhang, Li Jin, Zinan Liu, Shensi Wang, Shuchao Li, Stefanos Kollias (2024) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/pmc-11644511.txt.

Description

Yuying Shang and co-authors present a framework for filling gaps in knowledge graphs that mix text, images, and structure, in a 2024 paper. Completing such graphs means guessing missing facts. A subtle problem is missing modalities: many entities lack a picture or a description, and that absence can poison learning if ignored.

Their system, MERGE, builds three modality-specific graph networks that pass messages from neighbors, letting an entity borrow the missing modality's signal from its neighborhood. A low-rank tensor fusion then aligns the modalities at both structural and semantic levels, keeping the fused features interpretable. At prediction time, an interpolation re-ranking step rebalances how much each modality counts, without breaking each one's meaning. Tests on four public datasets showed the approach effective and robust, the authors report.

Catalog record

CREATORYuying Shang, Kun Fu, Zequn Zhang, Li Jin, Zinan Liu, Shensi Wang, Shuchao Li, Stefanos Kollias
YEAR2024
COLLECTIONPAPERS
FORMATTXT
FILE SIZE87 KB
VERSION1.0
LICENSECC BY 4.0
RIGHTSCC BY
SOURCEEurope PMC

Source: https://europepmc.org/articles/PMC11644511

Browse the full library · About the Tower of Babel