Col Mate : Contrastive Late Interaction and Masked Text for Multimodal Document Retrieval
Ahmed Masry, Megh Thakkar, Patrice Bechard, Sathwik Tejaswi Madhusudhan, Rabiul Awal, Shambhavi Mishra, Akshay Kalkunte Suresh, Srivatsava Daruru, Enamul Hoque, Spandana Gella, Torsten Scholak, Sai Rajeswar · 2025
Read Col Mate : Contrastive Late Interaction and Masked Text for Multimodal Document Retrieval by Ahmed Masry, Megh Thakkar, Patrice Bechard, Sathwik Tejaswi Madhusudhan, Rabiul Awal, Shambhavi Mishra, Akshay Kalkunte Suresh, Srivatsava Daruru, Enamul Hoque, Spandana Gella, Torsten Scholak, Sai Rajeswar (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2511-00903.txt.
Description
Ahmed Masry and colleagues present ColMate, a retrieval model for documents that mix text, layout, and images. Retrieval augmented generation needs such documents, yet most retrieval methods were copied from text only search, in how documents are encoded, trained, and scored.
ColMate changes all three parts. It pretrains with an OCR based objective, learns from a self supervised masked contrastive task, and scores matches with late interaction. That compares fine details between query and document rather than single summary vectors. On the ViDoRe V2 benchmark it improves over existing retrieval models by about three and a half percent, and it generalizes better to benchmarks outside its training domain.
Catalog record
| CREATOR | Ahmed Masry, Megh Thakkar, Patrice Bechard, Sathwik Tejaswi Madhusudhan, Rabiul Awal, Shambhavi Mishra, Akshay Kalkunte Suresh, Srivatsava Daruru, Enamul Hoque, Spandana Gella, Torsten Scholak, Sai Rajeswar |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 28 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2511-00903