Structured Semantic Information Helps Retrieve Better Examples for In-Context Learning in Few-Shot Relation Extraction
Aunabil Chakma, Mihai Surdeanu, Eduardo Blanco · 2026
Read Structured Semantic Information Helps Retrieve Better Examples for In-Context Learning in Few-Shot Relation Extraction by Aunabil Chakma, Mihai Surdeanu, Eduardo Blanco (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-20803.txt.
Introduction
Aunabil Chakma, Mihai Surdeanu, and Eduardo Blanco work on pulling facts out of text with very few examples. Their 2026 paper studies few-shot relation extraction. The task is to find links between entities in a sentence, such as who founded what. In-context learning lets a model copy the pattern from one given example. The catch is finding more good examples to show it.
The authors present several strategies to gather extra examples on their own. Their novel strategy picks examples by deep structure, not surface words. It compares the syntactic-semantic shape of each candidate with the one-shot example. A close match in structure should teach the model more. The work helps extraction systems that must start from almost no labeled data.
Catalog record
| CREATOR | Aunabil Chakma, Mihai Surdeanu, Eduardo Blanco |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 81 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2601-20803
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Structured Semantic Information Helps Retrieve Better Examples for In-Context Learning in Few-Shot Relation Extraction Aunabil Chakma, Mihai Surdeanu, Eduardo Blanco Abstract This paper presents several strategies to automatically obtain additional examples for in-context learning of one-shot relation extraction. Specifically, we introduce a novel strategy for example selection, in which new examples are selected based on the similarity of their underlying syntactic-semantic structure to the provided one-shot example. We show that this method results in complementary word choices and sentence structures when compared to LLM-generated examples. When these strategies are combined, the resulting hybrid system achieves a more holistic picture of the relations of interest than either method alone. Our framework transfers well across datasets (FS-TACRED and FS-FewRel) and LLM families (Qwen and Gemma). Overall, our hybrid selection method consistently outperforms alternative strategies and achieves state-of-the-art performance on FS-TACRED and strong gains on a customized FewRel subset. 1 Introduction Relation extraction (RE) is a core NLP task that identifies relations between entity pairs (e.g., the place of birth or employer of a person). Extracting relations is often a component of other tasks such as knowledge graph construction 24 . In turn, knowledge graphs coupled with retrieval-augmented approaches have been shown to improve reasoning 26 and reduce hallucinations 31 with LLMs, thus potentially benefiting any end-user application including question answering 35 . Traditional relation extraction approaches require many training examples per relation 15 . More recently, few-shot relation extraction (FSRE) has received substantial attention 22 ; 1 . In this setting, relations must be extracted given nothing but the relation name, a brief description, and a handful of examples. Because of this challenging setting, previous work obtains relatively low results (e.g., \approx 24 F1 on FS-TACRED). Existing approaches to FSRE include those grounded in prototypes 16 ; 10 and typically require training or fine-tuning (Section 2 ). Figure 1: Our approach to in-context learning for 1-shot relation extraction. Our method complements the (gold) 1-shot ("Support sentence" in the figure) with additional examples generated using shallower LLM-based methods (i.e., paraphrasing the gold example, or generating new examples based on the relation definition) or deeper methods using the underlying syntactic-semantic representations of candidate examples. When combining these strategies, the resulting few-shot relation extraction system achieves better results across two corpora, across several small language models. In-context learning (ICL) with LLMs ( 2 ) for FSRE is a natural choice yet it remains under-explored. In particular, the best strategy for example selection in RE remains unknown. To address this gap, in this paper we explore several strategies to automatically obtain additional examples for the relation of interest. In particular, we explore two main strategies: using LLMs to generate new examples, and retrieving examples from unannotated corpora based on their deeper syntactic-semantic similarity with the provided one-shot example. Figure 1 presents our strategies to obtain additional examples for one-shot relation extraction. Our main novelty is to automatically combine the one-shot gold example with additional examples that are: (a) generated with LLMs, or (b) retrieved from unannotated corpora using their syntactic-semantic similarity with the provided example. For LLM-based generation, we either paraphrase the gold example while preserving the subject, object, and relation, or generate new examples conditioned on the relation name and its natural-language definition, encouraging alternative lexicalizations and sentence structures. For the retrieval methods, we rely on lexico - syntactic rules , which denote structured representations that capture how the subject and object are connected in the sentence via their core syntactic, lexical, and semantic context; full details are given in Section 4.2 . Importantly, the semantic representations of these rules are learned in a self-supervised fashion that is agnostic to the relations of interest, similar to word embeddings 17 . Further, we emphasize diversity in the retrieved examples by: (a) clustering candidates using their semantic representations and selecting one representative example per cluster, and (b) using an LLM to select the most diverse examples from a pool of example candidates created with the above strategies. The main contributions of this paper are: • Several strategies to obtain additional examples for FSRE that range from shallower (e.g., LLM paraphrasing) to deeper (i.e., using representations of the syntactic-semantic structures of the texts of interest). • Methods to emphasize example diversity (i.e., diverse word choice and sentence structures) from a pool of candidate examples constructed using the above strategies. We investigate two directions, one based on the explicit clustering of examples and selecting one representative example per cluster, and a second based on prompt engineering. • Demonstrating that a hybrid approach balancing similarity and diversity obtains new state-of-the-art (SOTA) performance for FSRE, especially when smaller language models are used. 2 Related Work Early approaches and datasets for RE were designed for supervised learning 15 ; 5 . These approaches are effective when trained on hundreds or thousands of examples per relation 19 ; 38 . However, they rarely generalize across domains (i.e., same relation types expressed in documents from different sources) and can only identify at inference time the relations they were trained on. Distant supervision 18 has been explored to eliminate manual annotations, but other limitations remain, e.g., noise in the synthetic training data. Few-Shot Relation Extraction Most modern FSRE methods bypass the need to know the relations to be extracted at inference time and avoid manual annotations beyond a handful examples per relation. Representative approaches include prototype-based classification using aggregated relation representations ( 16 ; 10 ) . Embedding-based nearest-neighbor methods are also widely used ( 22 ) . While effective, these methods lack flexibility under relation or domain shifts. In contrast, neuro-symbolic approaches emphasize explicit relational cues ( 27 ; 25 ) , which improves their in- and out-of-domain performance. These properties motivate our approach to retrieve examples grounded in syntactic-semantic representations. While we are not the first to use in-context learning and LLMs for FSRE ( 14 ; 36 ) , we are the first to focus on improving ICL for FSRE with several strategies for example selection that improve both similarity and diversity. Unlike previous work, we demonstrate that our approach transfers across benchmarks and LLMs without needing any changes (e.g., prompt engineering). 1 1 1 We do not compare our results with 14 and 36 because they work with customized few-shot versions of TACRED and FewRel for which the exact few-shot evaluation setups are not publicly available. Example selection for In-Context Learning Outside RE, prior work has studied example selection for ICL for several NLP tasks ( 12 ; 4 ; 23 ; 32 ) . Existing approaches mainly rely on embedding-based similarity or learning-based selection using task supervision, model feedback, and diversity-aware criteria. These methods improve in-context performance for tasks such as semantic parsing, question answering, and text classification. In contrast, we leverage syntactic-semantic representations to guide example retrieval, an unexplored line of research in FSRE that moves beyond surface-level similarity by explicitly trading off relational similarity and contextual diversity. 3 The Few-Shot Relation Extraction Task Few-shot relation extraction is formulated as an episodic N -way k -shot classification task. A relation holds between two entities: the subject and object . Each episode consists of a k -shot learning instance defined by (a) a set of candidate relations \mathcal{R}=\{r_{1},\ldots,r_{N}\} along with