Derivation Prompting: A Logic-Based Method for Improving Retrieval-Augmented Generation

Ignacio Sastre, Guillermo Moncecchi, Aiala Rosá · 2026

Read Derivation Prompting: A Logic-Based Method for Improving Retrieval-Augmented Generation by Ignacio Sastre, Guillermo Moncecchi, Aiala Rosá (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-14053.txt.

Introduction

Sastre, Moncecchi and Rosa offer Derivation Prompting, a logic-based aid for retrieval-augmented generation in question answering. Language models often hallucinate or reason wrongly on knowledge-heavy, niche tasks. The method derives conclusions from starting hypotheses by applying set rules step by step. The result is a derivation tree that can be read and checked, giving more control over what the model writes. A case study cuts unacceptable answers well below those of plain RAG and long-context methods.

Catalog record

CREATORIgnacio Sastre, Guillermo Moncecchi, Aiala Rosá
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE25897
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2605.14053

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Derivation Prompting: A Logic-Based Method for Improving Retrieval-Augmented Generation Ignacio Sastre, Guillermo Moncecchi, Aiala Rosá Abstract The application of Large Language Models to Question Answering has shown great promise, but important challenges such as hallucinations and erroneous reasoning arise when using these models, particularly in knowledge-intensive, domain-specific tasks. To address these issues, we introduce Derivation Prompting, a novel prompting technique for the generation step of the Retrieval-Augmented Generation framework. Inspired by logic derivations, this method involves deriving conclusions from initial hypotheses through the systematic application of predefined rules. It constructs a derivation tree that is interpretable and adds control over the generation process. We applied this method in a specific case study, significantly reducing unacceptable answers compared to traditional RAG and long-context window methods. 1 1 1 Repo with all prompts: https://github.com/nsuruguay05/derivation-prompting 1 Introduction Question Answering (QA) has improved substantially with the advent of Large Language Models (LLMs). However, these models face important challenges, particularly in knowledge-intensive, domain-specific tasks, such as hallucinations and faulty reasoning [ 13 , 5 ] . The Retrieval-Augmented Generation (RAG) framework addresses these limitations by retrieving the most relevant document chunks from a trusted domain-specific document base, and grounding the LLM’s generation on the retrieved information [ 2 ] . Figure 1: Schematic illustration of a derivation tree constructed using derivation prompting. Substantial work has been done to improve the reasoning ability of LLMs [ 3 ] . Techniques like Chain-of-Thought (CoT) [ 15 ] have reliably improved performance across various tasks, including QA. However, these techniques do not explicitly define how the model should reason, as there are no restrictions on how the intermediate reasoning steps should be constructed. In this work, we propose Derivation Prompting, an alternative approach for the generation step in the RAG framework inspired by logical derivations. In this method, a conclusion is inferred from initial hypotheses by applying well-defined rules to transform and/or combine these hypotheses. This novel approach offers some advantages over existing methods, mainly: • Interpretability: The method not only generates a final answer but also produces a tree structure, referred to as a derivation (see Figure 1 ). Each node in the derivation represents the application of an easily interpretable rule that transforms some of its children. This structure provides a straightforward way to identify errors the model could have made and to understand how it arrived at the final answer. • Controlled generation: By generating the answer through the sequential application of predefined rules, this method provides a clearer reasoning path for the model to follow. This reduces hallucinations and faulty reasoning while ensuring that the generated answers remain grounded in the information from the documents. The paper is structured as follows: Section 2 presents related work. Section 3 provides a detailed explanation of Derivation Prompting. Section 4 describes the case study conducted. Section 5 explains the evaluation method used. Section 6 presents the results and analysis. Finally, Section 7 contains the conclusions and outlines future work. 2 Related work Retrieval Augmented Generation Derivation Prompting, as proposed in this work, is a prompting technique applied within the context of Retrieval-Augmented Generation (RAG). RAG augments LLMs with a retrieval component that recovers the most relevant information from an external knowledge base [ 8 , 4 , 11 ] . The naive RAG paradigm consists of two main steps: retrieval and generation. First, documents are segmented into smaller units, referred to as chunks, which are subsequently converted into vector representations. Upon a user query, during the retrieval step the query is converted to its vector representation, and similarity scores between the query and the indexed chunks are computed. The top k chunks are retrieved and used as context in the prompt for the generation step [ 2 ] . An alternative to using vector representations is to employ Cross-Encoder models that directly process each chunk with the query and return a similarity score [ 9 , 10 ] . While this approach tends to yield better results, it is significantly less compute-efficient, as it requires as many model inferences per query as number of chucks we have, compared to only one inference when using sentence embeddings. Prompting techniques for enhancing reasoning Substantial work has been done to improve the reasoning ability of LLMs [ 3 ] . Chain-of-Thought (CoT) [ 15 ] involves prompting the model to generate a coherent series of intermediate reasoning steps that lead to the final answer. Few-Shot prompting [ 1 ] is applied and a chain-of-thought is added to each example. They show that sufficiently large LLMs can generate these reasoning chains, yielding promising results in arithmetic, commonsense, and symbolic reasoning tasks. The Tree of Thoughts (ToT) framework [ 17 ] is an evolution of CoT that enables models to explore several different reasoning paths. In this approach, reasoning is conceptualized as searching through a tree, where each node represents a thought. This framework addresses some limitations of CoT, such as the inability to explore different continuations within the same reasoning chain or to backtrack when incorrect conclusions are reached. While these methods significantly improve performance on various tasks, there is no control over how each thought is generated in the chain, as there is no systematic methodology the model has to follow. This lack of control can lead to erroneous reasoning and susceptibility to hallucinations, which are well-known problems when working with LLMs [ 13 , 5 ] . Logic and LLMs Similar to Derivation Prompting, some works explore combining classical logic with prompting techniques to improve reasoning. The Logical Thoughts (LoT) prompting framework [ 18 ] uses logical equivalence, expressing premises in various logically equivalent forms to encourage the exploration of different solutions. This is achieved by incorporating a verification step for each thought, where an explanation is generated for both the thought as-is and its logical negation. The LLM is then tasked to decide between the two. Symbolic CoT (SymbCoT) [ 16 ] is another proposed method that involves four LLM modules: (i) Translator: translating premises and the question to First-Order Logic formulas, (ii) Planner: dividing the original problem into smaller subproblems and developing a step-by-step plan, (iii) Solver: deriving the answer through a logical inference process and (iv) Verifier: validating the correctness of the translations and the Solver’s output. 3 Derivation prompting This technique focuses on the generation step of the Retrieval-Augmented Generation (RAG) framework. It relies on the premise that the expected answer to a given query must be obtained by combining and/or transforming the most relevant information extracted from a document base, since our objective is to rely only on the information available in the documents and not on the information the model could have learned on its training phase. The idea for this technique is inspired by how a derivation tree in propositional logic is constructed. In this context, a conclusion \varphi is derived from a set of premises or hypotheses \Gamma=\{\delta_{1},\dots,\delta_{n}\} . We denote \Gamma\vdash\varphi if such a derivation exists. The class of derivations forms an inductively defined set characterized by a list of inference rules that explicitly state how to derive new conclusions from existing ones. These rules operate by systematically applying logical operations to the premises to construct a tree where each

Read the complete text