CRE-T1 Preview Technical Report: Beyond Contrastive Learning for Reasoning-Intensive Retrieval
Guangzhi Wang, Yinghao Jiao, Zhi Liu · 2026
Read CRE-T1 Preview Technical Report: Beyond Contrastive Learning for Reasoning-Intensive Retrieval by Guangzhi Wang, Yinghao Jiao, Zhi Liu (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-17387.txt.
Introduction
Wang, Jiao, and Liu propose T1, a generative retrieval model for reasoning-heavy search. Contrastive models freeze relevance into fixed vectors, which breaks when wording differs. T1 instead writes short reasoning paths for each query before it scores documents. A three-stage training plan ends with reinforcement learning to sharpen those paths. On the BRIGHT benchmark, the 4B model beat larger contrastive models overall, they report.
Catalog record
| CREATOR | Guangzhi Wang, Yinghao Jiao, Zhi Liu |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 35434 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2603.17387
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
CRE-T1 Preview Technical Report: Beyond Contrastive Learning for Reasoning-Intensive Retrieval Guangzhi Wang, Yinghao Jiao, Zhi Liu Abstract The central challenge of reasoning-intensive retrieval lies in identifying implicit reasoning relationships between queries and documents, rather than superficial semantic or lexical similarity. The contrastive learning paradigm is fundamentally a static representation consolidation technique: during training, it encodes hierarchical relevance concepts into fixed geometric structures in the vector space, and at inference time it cannot dynamically adjust relevance judgments according to the specific reasoning demands of each query. Consequently, performance degrades noticeably when vocabulary mismatch exists between queries and documents or when implicit reasoning is required to establish relevance. This paper proposes Thought 1 (T1), a generative retrieval model that shifts relevance modeling from static alignment to dynamic reasoning. On the query side, T1 dynamically generates intermediate reasoning trajectories for each query to bridge implicit reasoning relationships and uses <emb_token> as a semantic aggregation point for the reasoning output. On the document side, it employs an instruction + text + <emb_token> encoding format to support high-throughput indexing. To internalize dynamic reasoning capabilities into vector representations, we adopt a three-stage training curriculum and introduce GRPO in the third stage, enabling the model to learn optimal derivation strategies for different queries through trial-and-error reinforcement learning. On the BRIGHT benchmark, T1-4B exhibits strong performance under the original query setting, outperforming larger models trained with contrastive learning overall, and achieving performance comparable to multi-stage retrieval pipelines. The results demonstrate that replacing static representation alignment with dynamic reasoning generation can effectively improve reasoning-intensive retrieval performance. 1 Introduction Since the popularization of bi-encoder architectures such as Sentence-BERT [ 12 ] , contrastive learning [ 4 ] has become the standard paradigm for training dense vector models. However, in reasoning-intensive retrieval tasks, merely aligning “similarity” is often insufficient to capture “relevance.” The key challenge in such tasks is that the evidence supporting an answer frequently exhibits vocabulary mismatch with the query, requiring implicit causal, conditional, or multi-hop reasoning relationships to establish connections. Contrastive learning typically uses the representational distance between positive and negative samples as the supervisory signal, which tends to reward lexical matching and shallow semantic consistency while exhibiting noticeable performance degradation in scenarios requiring implicit reasoning. Examining this limitation from a deeper perspective, it is not accidental. Contrastive learning is fundamentally a static, hierarchical representation consolidation process: during training, it encodes the hierarchical conceptual judgment of “relevant/irrelevant” into the geometric structure of the vector space through pre-constructed positive-negative sample pairs, completing a one-way mapping from concepts to latent variables. Once training converges, this mapping is frozen as a fixed prior in the model parameters—a pre-training variable—and at inference time the model can only retrieve within this fixed representation space, unable to dynamically adjust its relevance judgment strategy according to the specific reasoning demands of a new query. This problem is particularly prominent in reasoning-intensive benchmarks such as BRIGHT [ 14 ] : a large number of queries require the model to identify implicit reasoning relationships rather than simple keyword or shallow semantic matches, yet the static representation space consolidated by contrastive learning struggles to cover these implicit reasoning relationships that require per-query derivation. To address this challenge, we propose the retrieval model Thought 1 (T1). Its core idea is to replace static alignment with dynamic reasoning: without significantly increasing retrieval cost, the model first generates a reasoning trajectory for each query and then compresses the reasoning result into a retrievable vector, so that relevance judgments are grounded in explicit derivation rather than solely relying on static representations consolidated during training. To motivate this design choice, we draw on our perspective on agent cognition: the essence of human problem-solving under limited cognitive resources lies in temporalized sequential reasoning [ 9 ] . Therefore, “scaling computation along the temporal dimension” is a key lever for improving capability. Based on this argument, related approaches can be naturally divided into two complementary lines. The first is test-time scaling , which invests more computation at inference time in exchange for accuracy; our prior work Refine Thought (RT) [ 16 ] belongs to this category, iteratively refining semantic representations in hidden space through multiple forward passes. The second is train-time scaling , which consolidates dynamic reasoning capabilities into model parameters during training through stronger supervision or reinforcement learning; for example, DeepSeek-R1 [ 3 ] enhances logical capabilities by optimizing reasoning trajectories. T1 adopts reinforcement learning at training time as its core mechanism, enabling the model to learn the ability to dynamically generate reasoning paths conditioned on each query, thereby transforming relevance modeling in reasoning-intensive retrieval from static representation alignment into a dynamic reasoning generation process. Experiments demonstrate that T1 exhibits significant advantages in reasoning-intensive retrieval scenarios such as BRIGHT. The key lies in fusing the query understanding and text representation stages of the retrieval pipeline: after observing the query constraints, the model first reasons, then compresses both the reasoning and query semantics into a query vector, which is compared with document vectors via similarity computation to complete retrieval. To achieve this, T1 adopts an asymmetric architecture: the query side generates a limited-length reasoning sequence and aggregates global semantic information through <emb_token> , while the document side extracts the corresponding token’s representation via instructions to ensure large-scale index construction efficiency. We internalize the aforementioned reasoning pattern into model parameters through reinforcement learning during training, thereby improving reasoning-intensive relevance modeling without significantly increasing online overhead. On the BRIGHT evaluation benchmark, T1 at the 4B scale using only original queries outperforms the combination of “retrieval model + query rewriting,” significantly surpasses contrastive baselines and commercial embedding models, and matches the state-of-the-art “retrieval model + reranking” combination pipeline. 2 Related Work We define reasoning-intensive retrieval as tasks where the relevance between a query and its relevant documents depends on implicit reasoning chains (semantic, causal, or logical) rather than mere keyword overlap or shallow semantic similarity. Early and mainstream dense retrieval benchmarks (e.g., BEIR [ 15 ] , MTEB [ 10 ] ) primarily focus on Information Seeking queries, where answers typically exist explicitly within text passages, or queries and documents exhibit high semantic match in vector space. The recent BRIGHT benchmark [ 14 ] further systematizes retrieval complexity into three levels: Level 1 relies on lexical matching, Level 2 relies on semantic matching, and Level 3 requires the model to locate evidence through implicit reasoning relationships. Compared with the first two levels, Level 3 tasks better reveal differences in models’ reasoning-based relevance modeling capabilities. For instance, in BRIGHT, some models that lead on MTEB (e.g., BGE [ 18 ] ) exhibit significant drops in nDCG@10, highlighting the gap between semantic matching evaluation and reasoning-intensive relevance. Existing work on reasoning-intensive retrieval can be broadly categorized into three technical lines based on the source of training signals and where reasoning augmentation occurs. The first line addresses reasoning signal sparsity from the perspective of data and supervision : it enhances the model’s representation learning on complex contexts through synthetic reasoning data or structured supervision, with representative works including ReasonIR [ 13 ] , RaDeR [ 2 ] , and ReasonEmbed [ 1 ] . The second line targets retrieval pipeline decomposition , introducing reasoning augmentation at different stages: on the query