Hybrid Pooling with LLMs via Relevance Context Learning DOI: XXXXXXX.XXXXXXX Conference: Make sure to enter the correct conference title from your rights confirmation emai; July 20–24, 2026; Melbourne, Australia ISBN: 978-1-4503-XXXX-X/18/06 CCS: Information systems Test collections CCS: Information

David Otero, Javier Parapar · 2026

Read Hybrid Pooling with LLMs via Relevance Context Learning DOI: XXXXXXX.XXXXXXX Conference: Make sure to enter the correct conference title from your rights confirmation emai; July 20–24, 2026; Melbourne, Australia ISBN: 978-1-4503-XXXX-X/18/06 CCS: Information systems Test collections CCS: Information by David Otero, Javier Parapar (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-08457.txt.

Description

Judging whether search results are relevant is essential for testing retrieval systems, yet human labeling is slow and costly. David Otero and Javier Parapar ask whether language models can judge better when they first learn what relevance means for a topic. Their arXiv paper introduces Relevance Context Learning, a two-model setup that makes topic criteria explicit.

An instructor model reads judged query-document pairs and writes a narrative of what counts as relevant for that topic. An assessor model then uses that narrative to judge new pairs. In a hybrid pooling test, humans judge a shallow pool while models label the rest. The method beats zero-shot prompting and standard in-context learning consistently. Readers get a smarter way to spend scarce human judgments.

Catalog record

CREATORDavid Otero, Javier Parapar
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE52 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2602-08457

Browse the full library · About the Tower of Babel