Joint Fusion and Encoding: Advancing Multimodal Retrieval from the Ground Up

Lang Huang * Qiyu Wu, Zhongtong Miao, Toshihiko YamasakiThe University of Tokyo, Tokyo, Japan · 2025

Read Joint Fusion and Encoding: Advancing Multimodal Retrieval from the Ground Up by Lang Huang * Qiyu Wu, Zhongtong Miao, Toshihiko YamasakiThe University of Tokyo, Tokyo, Japan (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2502-20008.txt.

Description

Lang Huang and co-authors present a new way to search with both pictures and words in this 2025 paper. Most search tools code pictures and text apart, then join the results late. That late join can miss fine links between what a query says and what a picture shows.

Their method fuses the two kinds of cues from the start, so each side shapes the other early. Training runs in two stages, first adapting the model and then refining it. The goal is search that reads complex queries more fully. Readers get a clear case for building fusion in from the ground up.

Catalog record

CREATORLang Huang * Qiyu Wu, Zhongtong Miao, Toshihiko YamasakiThe University of Tokyo, Tokyo, Japan
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE37 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2502-20008

Browse the full library · About the Tower of Babel