MIMO: Multilingual Information Retrieval via Monolingual Objectives
Youngjoon Jang, Seongtae Hong, Heuiseok Lim · 2026
Read MIMO: Multilingual Information Retrieval via Monolingual Objectives by Youngjoon Jang, Seongtae Hong, Heuiseok Lim (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-31171.txt.
Introduction
Jang, Hong and Lim improve search where queries and documents mix languages. Existing embeddings mainly serve one language at a time and lose accuracy when languages cross, while plain contrastive training trades alignment against uniformity. Their two-stage MIMO anchors learning in a strong English teacher space, first by distillation, then by joint distillation and contrastive training. The result beats cross-lingual baselines and rivals larger off-the-shelf models.
Catalog record
| CREATOR | Youngjoon Jang, Seongtae Hong, Heuiseok Lim |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 53313 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2605.31171
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
MIMO: Multilingual Information Retrieval via Monolingual Objectives Youngjoon Jang, Seongtae Hong, Heuiseok Lim Abstract Multilingual Information Retrieval (MLIR) reflects real-world search environments in which queries and relevant documents may appear in different languages within a mixed-language corpus. However, existing embedding models are primarily optimized for Multi-Monolingual retrieval and their performance often degrades in MLIR settings. Moreover, directly applying conventional contrastive learning to MLIR can exacerbate language clustering and expose a trade-off between cross-lingual alignment and embedding uniformity. To address these limitations, we propose MIMO : M ultilingual I nformation Retrieval via M onolingual O bjectives, a two-stage framework that uses a stable English semantic space from a high-performing teacher model as an anchor. MIMO first initializes the student model’s cross-lingual alignment through knowledge distillation, and then jointly optimizes distillation and cross-lingual contrastive learning to improve retrieval discrimination while preserving alignment. Extensive experiments show that MIMO consistently outperforms existing cross-lingual training baselines across various MLIR and Multi-Monolingual benchmarks. MIMO also remains competitive with off-the-shelf models of similar or larger parameter scales. Furthermore, our cross-lingual Alignment-Uniformity analysis clarifies the distinct roles of the two loss components and shows that their combination yields a favorable trade-off between alignment and uniformity. 1 Introduction As Information Retrieval (IR) technologies become globally ubiquitous, real-world user search environments are inevitably becoming multilingual ( 14 ; 27 ; 33 ; 29 ) . For instance, even when an English speaker issues a query in English, the most relevant and high-quality documents may be written in other languages, such as Korean, Chinese, or Spanish. This open-ended environment, where user queries and target documents can be in any arbitrary, distinct languages, is defined as Multilingual Information Retrieval (MLIR) ( 28 ) . To maximize user accessibility to information in the global web ecosystem, search systems capable of effectively handling these MLIR scenarios are essential ( 24 ; 32 ) . Figure 1: An illustration of the embedding space using xlm-roberta-large model trained with standard InfoNCE loss on monolingual pairs on the Belebele parallel dataset. (a) and (b) show monolingual retrieval scenarios where queries and corpus are in the same language. (c) and (d) demonstrate multilingual scenarios where corpus consist of mixed languages. Recently, advancements in dense retrieval based on multilingual Pre-trained Language Models (mPLMs) have significantly improved the performance of models handling multiple languages ( 40 ; 7 ; 1 ; 43 ; 11 ) . However, existing multilingual retrieval models, along with their training strategies and evaluation datasets, are predominantly optimized for Multi-Monolingual IR scenarios, where the query and document languages remain identical. While these models exhibit outstanding matching performance within individual languages, they reveal severe limitations in MLIR scenarios that require identifying relevant answers across language boundaries within a mixed-language corpus ( 26 ; 13 ) . The issues faced by conventional training methodologies in MLIR environments are clearly observed in Figure 1 . This figure visualizes queries and documents from the Belebele parallel dataset ( 4 ) in the embedding space, using an xlm-roberta-large encoder fine-tuned exclusively with the InfoNCE objective on monolingual query-positive pairs. As shown in (a) and (b), in Multi-Monolingual scenarios where the query and document languages are identical, the model successfully places the query and its positive document close together, yielding high cosine similarities of 0.86 and 0.83. Conversely, (c) and (d) illustrate MLIR scenarios where a single query is evaluated against a mixed-language corpus. Specifically, the model is tasked with retrieving from a mixed corpus where four semantically identical documents are translated into four different languages. In these cases, the data points are severely clustered by language rather than semantics. Even though the four documents possess identical meanings, their superficial linguistic differences cause the average cosine similarity to drop significantly to 0.73 and 0.67. This indicates that InfoNCE, a widely-used training method fails to achieve proper cross-lingual alignment, leading to the formation of language-specific islands and ultimately causing a degradation in MLIR performance. To resolve these issues, we propose MIMO (Multilingual Information Retrieval via Monolingual Objectives). MIMO utilizes a robust English embedding space from a high-performing teacher model as an absolute anchor. The student model initializes its cross-lingual alignment space through a Stage 1 knowledge distillation warmup, learning to map multilingual inputs into the teacher’s English semantic space. In the subsequent Stage 2, it jointly optimizes knowledge distillation and cross-lingual contrastive learning. Notably, while it was known from 37 that contrastive learning improves embedding uniformity, our study indicates that knowledge distillation drives the alignment of the embedding space in cross-lingual environments. Building upon this discovery, MIMO effectively separates the roles of each loss function, providing an effective operating point to balance the alignment-uniformity trade-off. In summary, the main contributions of this paper are as follows: • We propose MIMO, a two-stage training pipeline for MLIR that uses a stable English semantic anchor provided by a teacher model to combine knowledge distillation and cross-lingual contrastive learning for multilingual representation learning. • We extend the Alignment-Uniformity analysis to cross-lingual settings, clarifying the distinct roles of the distillation and contrastive objectives and showing that their combination provides a favorable trade-off between alignment and uniformity. • MIMO consistently outperforms existing cross-lingual training baselines on MLIR and multi-monolingual benchmarks. It also achieves highly competitive performance against off-the-shelf models of similar or larger parameter scales. 2 Related Works 2.1 Multilingual Retrieval Settings Multilingual retrieval environments can be defined in various ways depending on the language composition of the query and the corpus. Building upon the definitions discussed in the early Cross-Language Evaluation Forum (CLEF) and NeuCLIRBench, we categorize the environments of modern retrieval benchmarks as shown in Table 1 ( 6 ; 19 ) . Setting Query Language Corpus Language Monolingual IR L_{1} (fixed) L_{1} (fixed) Cross-Lingual IR (CLIR) L_{1} (fixed) L_{2} (fixed) Multi-Monolingual IR L L (same, but L varies) Multilingual IR (MLIR) L (any) L_{mixed} Table 1: Comparison of multilingual retrieval settings. L , L_{1} , and L_{2} denote specific languages. Despite MLIR being the most realistic setting for global web search, recent benchmarks predominantly focus on Multi-Monolingual IR, where query and document languages are paired identically ( 41 ; 7 ; 1 ; 11 ) . Even advanced mixed-corpus benchmarks like NeuCLIRBench restrict queries strictly to English ( 19 ) , leaving a critical gap in evaluating any-to-any language retrieval. 2.2 Training Methods Contrastive Learning The InfoNCE loss is the standard dense retrieval objective, optimizing models by pulling a query closer to a positive document while pushing away in-batch negatives ( 15 ) . However, existing multilingual retrieval models predominantly train on monolingual query-document pairs ( 3 ) . Therefore, models trained solely with this formulation possess a fundamental limitation in effectively bridging the semantic gap between texts written in different languages. Cross-Lingual Alignment To enhance cross-lingual alignment, 8 proposed a Cross-lingual Contrastive (XLCO) Objective that directly utilizes query-document pairs in different languages. In a similar vein, 38 proposed the LaKDA loss, which minimizes the Symmetric KL Divergence between the similarity probability distributions of semantically parallel query pairs over a batch of documents. However, these methodologies share a common limitation: they remain confined to relative alignment, attempting to match multilingual representations internally within the student model. Due to the absence of a stable absolute anchor,