Joint Fusion and Encoding: Advancing Multimodal Retrieval from the Ground Up
Lang Huang * Qiyu Wu, Zhongtong Miao, Toshihiko YamasakiThe University of Tokyo, Tokyo, Japan · 2025
Read Joint Fusion and Encoding: Advancing Multimodal Retrieval from the Ground Up by Lang Huang * Qiyu Wu, Zhongtong Miao, Toshihiko YamasakiThe University of Tokyo, Tokyo, Japan (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2502-20008.txt.
Description
Lang Huang and co-authors present a new way to search with both pictures and words in this 2025 paper. Most search tools code pictures and text apart, then join the results late. That late join can miss fine links between what a query says and what a picture shows.
Their method fuses the two kinds of cues from the start, so each side shapes the other early. Training runs in two stages, first adapting the model and then refining it. The goal is search that reads complex queries more fully. Readers get a clear case for building fusion in from the ground up.
Catalog record
| CREATOR | Lang Huang * Qiyu Wu, Zhongtong Miao, Toshihiko YamasakiThe University of Tokyo, Tokyo, Japan |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 37 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2502-20008