MeVer at CheckThat! 2026: Cluster-Aware Hard-Negative Mining for Multilingual Scientific-Source Retrieval

Juli Bakagianni, Symeon Papadopoulos · 2026

Read MeVer at CheckThat! 2026: Cluster-Aware Hard-Negative Mining for Multilingual Scientific-Source Retrieval by Juli Bakagianni, Symeon Papadopoulos (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-24236.txt.

Introduction

Bakagianni and Papadopoulos tackle multi-language paper search, matching short, casual claims to their papers, where related papers act as hard decoys. Their CheckThat 2026 entry studies hard-negative training for multi-stage search. Cluster negatives from candidate sets train dense search and second passes better. Local cluster negatives favour hits, while broader meaning negatives give reach and stable second passes across tongues. Constrained sorting prompts prove the soundest final proof picker. The system, dense searcher plus multi-language second pass plus a tie-breaker, ranked sixth of 37 entries.

Catalog record

CREATORJuli Bakagianni, Symeon Papadopoulos
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE42552
LICENSEhttp://creativecommons.org/licenses/by-sa/4.0/
RIGHTSCC BY SA
SOURCEarXiv

Source: https://arxiv.org/abs/2605.24236

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

MeVer at CheckThat! 2026: Cluster-Aware Hard-Negative Mining for Multilingual Scientific-Source Retrieval Juli Bakagianni, Symeon Papadopoulos Abstract Identifying the scientific source behind a social media claim requires matching short, informal, and often multilingual claims against large collections of scientific publications, where semantically related papers may act as challenging distractors or false negatives during training. We present our submission to CheckThat! 2026 Task 1 on multilingual scientific-source retrieval, focusing on how hard-negative mining should be adapted to multi-stage retrieval pipelines for scientific-source retrieval. We propose cluster-aware hard-negative mining strategies that exploit the semantic structure of retrieved candidate pools in order to construct more informative training negatives for dense retrieval and reranking. Our experiments show that different hard-negative structures induce different retrieval behaviors. Localized cluster negatives tend to favor precision-oriented retrieval, whereas broader non-gold semantic negatives provide stronger candidate coverage and more consistent reranking performance across languages. We further study multiple LLM-based evidence-selection formulations, including direct classification, pairwise comparison, and listwise reranking prompts, and find that constrained classification prompts provide the most reliable final document selection. The final system combines a dense retriever, a multilingual cross-encoder reranker, and a selective LLM-based disagreement resolver, ranking 6th among 37 submissions in the shared task evaluation. Overall, our results suggest that hard-negative mining should be treated as a stage-aware design problem rather than as a single retrieval optimization strategy. Keywords i nformation retrieval; social media retrieval; hard-negative mining; scientific-source retrieval; cross-encoder reranking; large language models 1 Introduction Scientific claims are frequently discussed on social media through informal references to studies, reports, or “research findings” without explicit citations or links to the underlying publication. Automatically identifying the referenced scientific paper is therefore an important problem for scientific fact-checking, evidence tracing, and misinformation analysis 1 . However, this retrieval setting is particularly challenging because short and loosely phrased social media claims must be matched against large collections of scientific articles written in formal academic language. The problem becomes even more difficult in multilingual settings, where claims may appear in different languages while the scientific collection remains primarily English. Moreover, since the dataset provides only a single annotated source paper per claim, semantically related but unlabeled papers may act as false negatives during retrieval and reranking training. CheckThat! Task 1 formulates this problem as a multilingual scientific-source retrieval task. Given a social media claim in English, German, or French, the goal is to retrieve the referenced scientific paper from a shared collection of scientific publications 7 . The task combines known-item retrieval with semantic evidence matching: the correct paper may differ from competing candidates only in subtle methodological or semantic details, requiring systems to balance broad candidate coverage with fine-grained semantic discrimination. Modern retrieval systems for such tasks are commonly organized as multi-stage pipelines. First-stage dense retrievers retrieve candidate documents using learned embedding similarity representations 2 ; 9 . Cross-encoder rerankers then refine the ranking through full query–document interaction modeling 3 ; 6 . More recently, large language models (LLMs) have also been explored as reasoning-based rerankers or judges capable of comparing a small set of candidate documents and selecting the best supporting evidence 8 ; 5 . These stages address complementary objectives: retrieval emphasizes candidate-set coverage, reranking emphasizes local semantic discrimination, and LLM-based judgment targets difficult final selection cases where multiple candidates remain plausible. In this work, we focus on how hard negative ( hn ) mining during model training should be adapted to such a multi-stage retrieval pipeline. Existing hn approaches for contrastive retriever and reranker training typically select negatives based on retrieval rank or nearest-neighbor similarity 9 ; 10 . However, retrieved candidate pools often exhibit internal semantic structure: some documents form highly localized neighborhoods around the gold paper, while others act as broader topical distractors. We therefore investigate whether clustering retrieved candidates can provide a more principled mechanism for selecting informative training negatives for both retriever and reranker fine-tuning. Our central hypothesis is that different hn structures induce different retrieval behaviors across multi-stage pipelines. Broader semantic negatives may improve downstream candidate coverage, whereas highly localized negatives may sharpen fine-grained semantic discrimination between closely related scientific papers. We therefore study how different cluster-aware hn strategies affect both retrieval and reranking stages. To study this question, we introduce cluster-aware hn mining strategies that sample negatives from the gold-document cluster, neighboring clusters, or non-gold semantic regions of the retrieved space. Beyond retrieval and reranking, we also investigate how LLMs should operate as selective final-stage judges. Rather than applying the LLM universally, we use it only on disagreement cases between the retriever and reranker. We further study multiple prompting and decision formulations for LLM-based evidence selection, including direct classification, pairwise comparison, permutation-style ranking inspired by RankGPT 8 , shuffled candidate ordering, and committee-style voting. Our experiments show that simple constrained classification prompts are more reliable than more elaborate ranking-style formulations in this retrieval setting. Although multilingual handling is not the primary focus of this work, our experiments also reveal an interesting interaction between translation strategy and judge model family. Translation into English consistently improves first-stage retrieval and reranking for non-English queries, likely because the scientific collection itself is English. However, at the LLM judgment stage, GPT-family models benefit from the original-language claim formulation, whereas Llama-based judges perform better with translated claims. This suggests that optimal query representation depends not only on the pipeline stage but also on the downstream judge model and the candidate distribution produced by earlier retrieval stages. The paper is organized around the following research questions: • Can cluster-aware hn mining improve retrieval candidate coverage and reranking effectiveness in multilingual scientific-source retrieval? • Do retrieval and reranking stages benefit from different forms of semantic hn ? • How should LLMs be used as final-stage judges in retrieve-then-rerank pipelines, and which prompting formulations are most effective for evidence selection? Our final system combines a dense retriever and a cross-encoder reranker trained with cluster-aware hn and a selective GPT-5.5 disagreement resolver. Across experiments, we find that non-gold-cluster negatives provide the strongest retrieval backbone for reranking-oriented pipelines, while localized cluster negatives are more effective for sharpening retrieval-only precision. We further show that selective LLM intervention is most effective when restricted to disagreement cases and formulated as constrained classification rather than free-form ranking. Together, these results suggest that hn mining should be treated as a stage-aware design problem rather than a single retrieval optimization strategy. 2 Related Work Dense retrieval has become a standard approach for semantic document retrieval using contrastive representation learning 2 ; 9 . A central component of dense retriever training is hn mining, where models are trained to distinguish relevant documents from semantically similar distractors. Prior work has explored Approximate Nearest Neighbor Negative Contrastive Learning (ANCE) 9 , BM25-based negatives 2 , denoised hn 10 , and curriculum-style training strategies. However, most existing approaches define negative difficulty primarily through retrieval rank or embedding similarity, without explicitly modeling the semantic structure of the retrieved candidate space. Retrieve-then-rerank pipelines further improve retrieval quality by combining dense retrievers with cross-encoder rerankers 3 ; 6 . More recently, LLMs have also been studied for ranking or reranking tasks 8 ; 5 , including permutation-based ranking and pairwise prompting strategies. Existing work has shown that LLMs can act as effective rerankers

Read the complete text