GLiDRE: Generalist Lightweight model for Document-level Relation Extraction
Robin Armingaud, Romaric Besançon · 2025
Read GLiDRE: Generalist Lightweight model for Document-level Relation Extraction by Robin Armingaud, Romaric Besançon (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2508-00757.txt.
Introduction
Robin Armingaud and Romaric Besançon present a model from two thousand twenty-five for finding relations between entities across a whole document. Relation extraction is a core task in language processing, but at document level the clues are spread across sentences. Trained models do well with plenty of data, yet their behavior with little data was poorly studied.
Their model, called GLiDRE, is compact and built to work when training data are scarce. It targets both low-resource supervised training and few-shot settings, where only a handful of examples are given. In both kinds of tests it beat existing methods under tight data limits. The authors say the code will be made public, which aids reuse.
Catalog record
| CREATOR | Robin Armingaud, Romaric Besançon |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 35 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2508-00757
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
GLiDRE: Generalist Lightweight model for Document-level Relation Extraction Robin Armingaud, Romaric Besançon Abstract Relation Extraction (RE) is a fundamental task in Natural Language Processing, and its document-level variant poses significant challenges, due to the need to model complex interactions between entities across sentences. Current approaches, largely based on the ATLOP architecture, are commonly evaluated on benchmarks like DocRED and Re-DocRED. However, their performance in zero-shot or few-shot settings remains largely underexplored due to the task’s complexity. Recently, the GLiNER model has shown that a compact NER model can outperform much larger Large Language Models. With a similar motivation, we introduce GLiDRE, a new model for document-level relation extraction that builds on the key ideas of GliNER. We benchmark GLiDRE against state-of-the-art models across various data settings on the Re-DocRED dataset. Our results demonstrate that GLiDRE achieves state-of-the-art performance in few-shot scenarios. Our code is publicly available 1 1 1 https://github.com/robinarmingaud/glidre . 1 Introduction Document-level Relation Extraction (DocRE) is a challenging task in natural language processing that involves identifying relationships between entities across multiple sentences. In contrast, traditional sentence-level relation extraction, evaluated on supervised datasets like TACRED ( Zhang et al. 2017 ) and SemEval-2010 Task 8 ( Hendrickx et al. 2010 ) or few-shot benchmarks such as FewREL ( Han et al. 2018 ) and Wiki-ZSL ( Chen and Li 2021 ) , typically considers a single entity pair per instance, often without negative examples (i.e. sentences where no relation is present). These settings closely resemble relation classification rather than relation extraction. While recent works improve existing benchmarks, like FewRel 2.0 ( Gao et al. 2019 ) which addresses the negative examples issue, they still operate under the constraint of classifying a fixed pair of entities. DocRE presents a more realistic and complex scenario, especially relevant for real-world applications such as Information Extraction (IE) in biomedical or industrial texts, where relationships often span sentence boundaries and the entity pairs of interest are not pre-identified. The complexity of DocRE is due to the quadratic growth of negative pairs with the number of entities in a document. Evaluation is commonly conducted on datasets like DocRED ( Yao et al. 2019 ) and its revised version Re-DocRED ( Tan et al. 2022b ) , which reduces the issue of false negatives. These two datasets contain more complex documents with an average of 19 entities and approximately 390 candidate pairs per text, for a total of 3,053 annotated Wikipedia articles in the training set. State-of-the-art approaches to DocRE generally extend the model ATLOP ( Zhou et al. 2021 ) , which introduces Localized Context Pooling, using attention mechanisms to enhance relation representations, and an adaptive-thresholding loss that adjusts decision thresholds dynamically based on entity pairs. Subsequent work has extended ATLOP in various directions. For example, DREEAM ( Ma et al. 2023 ) uses evidences to enhance performance, while KD-DocRE ( Tan et al. 2022a ) uses axial attention to model interdependencies among entity pairs, adopts an adaptive focal loss to address class imbalance, and integrates knowledge distillation. Figure 1: Model architecture. GLiDRE uses a dual-encoder structure: a text encoder generates token embeddings, while a label encoder computes embeddings for relation types. An entity’s representation is formed by pooling the embeddings of its constituent tokens and mentions. For each entity pair, their representations are concatenated and processed by a feed-forward network before final scoring. The model can optionally incorporate a localized context pooling module, inspired by ATLOP, to enrich the relation representation. However, if recent works tackle sentence-level relation extraction in low-resource settings ( Boylan et al. 2025 , Li et al. 2024a , Lan et al. 2023 ), DocRE in similar settings remains largely unexplored. Although latest LLMs show strong capabilities in zero-shot settings for tasks such as NER and RE ( Sainz et al. 2024 , Zhou et al. 2023 , Wang et al. 2023 , Wei et al. 2023 ), their performance on zero-shot DocRE, especially on the DocRED or Re-DocRED dataset is still limited ( Li et al. 2023 , Xue et al. 2024 ). The recent success of GLiNER ( Zaratiana et al. 2024 ) has motivated this work. GLiNER deals with the task of NER as matching entity type representations to textual span representations in the latent space. This approach allows better representations than LLMs due to its use of bidirectional encoders and it solves the scalability issues of autoregressive models, surpassing much larger LLMs in zero shot settings. Moreover, as a zero-shot model, GLiNER is not restricted to a fixed set of relation labels and can generalize to arbitrary relation types at inference time. Based on this work, we introduce GLiDRE, a model inspired by the bi-encoder version of GLiNER 2 2 2 https://blog.knowledgator.com/meet-the-new-zero-shot-ner-architecture-30ffc2cb1ee0 , adapted for DocRE. Our contributions in this paper are : • We propose GLiDRE, a new and efficient model for document-level relation extraction, inspired by the core principles of GLiNER. We also adapt and test key components from ATLOP to improve relation representations. • We conduct extensive experiments on the Re-DocRED benchmark, demonstrating that GLiDRE achieves state-of-the-art results in challenging few-shot scenarios. • We demonstrate that smaller, specialized encoder models can offer an efficient alternative to larger LLMs for complex, document-level relation extraction and we present an up-to-date evaluation of these models on the Re-DocRED dataset in a zero-shot setting. 2 Methodology 2.1 Document Level Relation Extraction Let a document be represented as a sequence of tokens D . Within this document, there is a set of m pre-identified entities E=\{e_{1},e_{2},\dots,e_{m}\} , where each entity e_{i} is associated with one or more mentions in the text. The objective of DocRE is to identify valid relational triplets. Each triplet is of the form (e_{h},r,e_{t}) , where e_{h},e_{t}\in E are the head and tail entities respectively ( h can be equal to t ), and r is the relation that holds between them. Based on this definition, DocRE is as a multi-label classification task. 2.2 GLiDRE architecture GLiDRE extends the bi-encoder variant of GLiNER. This architecture employs two separate transformers, one for encoding the document and another for encoding relation labels, which offers several benefits : unlike the uni - encoder version, the bi-encoder approach avoids concatenating labels and document tokens into a single input. Instead, we encode the document and labels separately. This eliminates the need for special separation tokens and overcomes encoder context length limits, permitting an effectively unlimited label set. Moreover, label embeddings can be precomputed, accelerating inference. However, the bi - encoder architecture increases memory usage and lacks cross - attention between labels. As a result, it may struggle to disambiguate semantically similar labels. Label and word representations Label embeddings are obtained by mean-pooling over the token representations constituting label names. The computed label embeddings are passed through a two-layer feedforward network if the dimension of the label encoder’s output differs from the configured latent space dimension of GLiDRE. For document tokens, we adopt the standard strategy used in GLiNER and other standard NER models : for words split into subwords, we extract the representation of the first subword. Relation representations Given contextual embeddings H\in\mathbb{R}^{L\times D} from the document encoder, where L is the sequence length and D is the hidden