ConceptFormer: Towards Efficient Use of Knowledge-Graph Embeddings in Large Language Models
Joel Barmettler, Abraham Bernstein, Luca Rossetto · 2025
Read ConceptFormer: Towards Efficient Use of Knowledge-Graph Embeddings in Large Language Models by Joel Barmettler, Abraham Bernstein, Luca Rossetto (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2504-07624.txt.
Introduction
Joel Barmettler, Abraham Bernstein, and Luca Rossetto present ConceptFormer in two thousand twenty-five. Language models need world knowledge from knowledge graphs. Textifying graphs wastes tokens and strains context windows. Editing model internals is the other costly route.
ConceptFormer works in the model's own embedding space instead. Knowledge graph nodes become concept vectors injected at input. A lookup table maps each node to its vector. The language model itself stays frozen during training. Factual recall on GPT-2 jumped sharply in their tests. One vector alone beat graph text retrieval at a fraction of tokens.
Catalog record
| CREATOR | Joel Barmettler, Abraham Bernstein, Luca Rossetto |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 53 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2504-07624
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
ConceptFormer: Towards Efficient Use of Knowledge-Graph Embeddings in Large Language Models Joel Barmettler, Abraham Bernstein, Luca Rossetto Abstract . Retrieval Augmented Generation (RAG) has enjoyed increased attention in the recent past and recent advancements in Large Language Models (LLMs) have highlighted the importance of integrating world knowledge into these systems. Current RAG methodologies often modify the internal architecture of pre-trained language models (PLMs) or rely on textifying knowledge graphs (KGs), which is inefficient in terms of token usage. This paper introduces ConceptFormer , a new approach to augment LLMs with structured knowledge from KGs, such as Wikidata, without altering their internal structure or relying on textual input of KGs. ConceptFormer operates in the LLM embedding vector space, creating and injecting concept vectors that encapsulate the information of the KG nodes directly. Trained in conjunction with a frozen LLM, ConceptFormer generates a comprehensive lookup table that maps KG nodes to their respective concept vectors . The approach aims to enhance the factual recall capabilities of LLMs by enabling them to process these concept vectors natively, thus enriching them with structured world knowledge in an efficient and scalable manner. Our experiments demonstrate that the addition of concept vectors to GPT-2 0.1B substantially increases its factual recall ability (Hit@10) by up to 272% when tested on sentences from Wikipedia and up to 348% on synthetically generated sentences. Even injecting only a single concept vector into the prompt increases factual recall ability (Hit@10) by up to 213% on Wikipedia sentences, significantly outperforming RAG with graph textification while consuming 130x fewer input tokens. CCS: Computing methodologies Knowledge representation and reasoning CCS: Information systems Data encoding and canonicalization CCS: Information systems Language models 1. Introduction Figure 1. ConceptFormer enhances a prompt by extending the embedding vectors from the original prompt with learned concept vectors . Entity recognition and entity linking is used to detect an entity “Albert Einstein” (displayed in blue) in the original prompt and link it towards a large KG like Wikidata. ConceptFormer creates a vector embedding for the detected entity that is compatible with the LLMs input embedding space. The resulting concept vector (displayed in green) is shown to capture the essence of the entity far better than the original token embedding vectors alone, leading to more knowledgeable output generated by the LLM (displayed in red). ConceptFormer enhances a prompt by extending the embedding vectors from the original prompt with learned concept vectors. Entity recognition and entity linking is used to detect an entity ``Albert Einstein'' in the original prompt and link it towards a large KG like Wikidata. ConceptFormer creates a vector embedding for the detected entity that is compatible with the LLMs input embedding space. The resulting concept vector (displayed in green) is shown to capture the essence of the entity far better than the original token embedding vectors alone, leading to more knowledgeable output generated by the LLM. Large Language Models (LLMs) have demonstrated exceptional potential in various natural language processing tasks, including conversational agents, summarization, and information retrieval (IR). They are typically trained on large-scale, general-purpose (text) corpora and optimized via self-supervision objectives ( Li et al., 2021 ) . Through this pretraining, LLMs acquire substantial amounts of knowledge, which is implicitly stored within their model weights ( Zhou et al., 2023 ; Roberts et al., 2020 ) . However, such implicit storage can lead to inefficient knowledge retrieval and risks of outdated, biased, or incomplete information, posing key obstacles for retrieval-augmented generation (RAG) and knowledge-intensive IR tasks ( Lewis et al., 2020 ; Liska et al., 2022 ; Luu et al., 2022 ) . A common approach to enhance knowledge retention in LLMs is through corpus curation or expansion, which has shown promise in reducing hallucinations ( Gardent et al., 2017 ; Wang, 2020 ) . But simply improving or enlarging the training corpus does not always resolve deeper IR challenges such as domain-specific knowledge retrieval or real-time updates ( Touvron et al., 2023 ; Gunasekar et al., 2023 ) . Models still frequently hallucinate or omit key facts ( Pörner et al., 2019 ; Min et al., 2023 ) , especially in specialized domains where domain shifts are inevitable ( Wang et al., 2023a ) . Research has shown that growing the parameter count of LLMs can improve factual recall ( Lee et al., 2022 ) , but even large models like ChatGPT still struggle to accurately retrieve and articulate structured knowledge, e.g., from DBPedia ( Wu et al., 2023 ; Sun et al., 2023 ) . Such difficulties reflect broader IR concerns: a system’s inability to efficiently retrieve relevant concepts hampers downstream tasks like question answering, recommendation, and conversational search. To address these challenges in knowledge-intensive scenarios, Knowledge Graphs (KGs) ( Ji et al., 2022 ; Mitchell et al., 2018 ) have emerged as a valuable structured resource. KGs—such as Wikidata ( Vrandecic and Krötzsch, 2014 ) —capture accurate, up-to-date, and domain-rich factual information. Making this graph-based knowledge accessible to LLMs is a longstanding goal in IR research, especially for retrieval-augmented generation and question answering ( Yu et al., 2022 ; Abu-Salih, 2021 ) . Popular RAG-focused approaches typically rely on “ textification ” of graph edges or nodes ( Fatemi et al., 2023 ; Li et al., 2023 ) , concatenating them into the LLM prompt. While effective, this strategy consumes large portions of the model’s context window and can introduce noise ( Zheng et al., 2024 ) . Consider a conversational IR system that must answer a user’s query about a niche topic (e.g., “What type of mustache did Albert Einstein famously sport?”). If the system is forced to feed hundreds of tokens of textified KG neighbors into the LLM prompt for every entity, it risks hitting context limits or drowning in irrelevant details. An alternative is to embed this knowledge in a more compact form, seamlessly integrating it with the user query inside the LLM’s embedding space. Such a solution would not only save tokens but also yield more targeted retrieval results, echoing a core IR theme: balancing retrieval accuracy with efficiency. In this paper, we introduce ConceptFormer , a novel, token-efficient way to integrate KGs into any pre-trained LLM without modifying its internal architecture or retraining its core parameters. Rather than textifying knowledge, ConceptFormer injects concept vectors derived from the KG directly into the LLM’s input embedding space. Figure 1 illustrates how entity recognition links a text mention (“Albert Einstein”) to its corresponding Wikidata node, and ConceptFormer learns to produce a small set of dense concept vectors that augment or replace the naive token embeddings. These vectors natively encode the most relevant graph-neighborhood information, enabling the LLM to integrate structured knowledge with minimal context overhead. The contributions of this paper are threefold: (1) We present ConceptFormer , a flexible mechanism to embed KG nodes into the LLM prompt space, enabling IR scenarios that demand large-scale or domain-specific factual retrieval without altering the LLM architecture. (2) We introduce new datasets— Tri-REx , T-REx Bite , and T-REx Star —specifically designed to evaluate next-token and factual-recall tasks.These datasets facilitate measuring how well LLMs retrieve, re-rank, and generate entity-level facts. (3) We empirically demonstrate that ConceptFormer achieves up to 348% improvement in factual recall (Hit@10)