From Reasoning to Learning: A Survey on Hypothesis Discovery and Rule Learning with Large Language Models

Kaiyu He [email protected], Zhiyu Chen [email protected] · 2025

Read From Reasoning to Learning: A Survey on Hypothesis Discovery and Rule Learning with Large Language Models by Kaiyu He [email protected], Zhiyu Chen [email protected] (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-21935.txt.

Introduction

Kaiyu He and Zhiyu Chen present a 2025 survey of whether large language models can discover new knowledge, not just follow orders. Most work so far has sharpened instruction following and step by step deduction. Real discovery needs fresh hypotheses that can be tested and turned into rules.

The survey uses a classic three part frame for reasoning: guessing a cause, drawing out its consequences, and checking them against data. It sorts past work on forming, applying, and testing hypotheses with language models. The authors name the main wins and the open gaps. Their aim is a map for turning today's models into engines of discovery.

Catalog record

CREATORKaiyu He [email protected], Zhiyu Chen [email protected]
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE69 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2505-21935

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

From Reasoning to Learning: A Survey on Hypothesis Discovery and Rule Learning with Large Language Models Kaiyu He [email protected], Zhiyu Chen [email protected] Abstract Since the advent of Large Language Models (LLMs), efforts have largely focused on improving their instruction-following and deductive reasoning abilities, leaving open the question of whether these models can truly discover new knowledge. In pursuit of artificial general intelligence (AGI), there is a growing need for models that not only execute commands or retrieve information but also learn, reason, and generate new knowledge by formulating novel hypotheses and theories that deepen our understanding of the world. Guided by Peirce’s framework of abduction, deduction, and induction, this survey offers a structured lens to examine LLM-based hypothesis discovery. We synthesize existing work in hypothesis generation, application, and validation, identifying both key achievements and critical gaps. By unifying these threads, we illuminate how LLMs might evolve from mere “information executors” into engines of genuine innovation, potentially transforming research, science, and real-world problem solving. Reviewed on OpenReview: https://openreview.net/forum?id=d7W38UzUg0 1 Introduction One major pillar of human intelligence is the capacity to discover hypotheses and learning rules. We call this capability hypothesis discovery (or rule learning). Earlier AI systems struggled with it because formal symbolic methods lacked the commonsense background needed for inventive rule formation ( Yu et al., 2024a ) . Recent advances in natural language processing (NLP) have produced LLMs pretrained on extensive text corpora that embed substantial commonsense knowledge. These models now enable tasks that demand rich background knowledge, such as formulating new hypotheses and deriving novel conclusions. {forest} Figure 1: Taxonomy for Hypothesis Discovery with LLMs. Our survey categorizes work into four topics based on Peirce’s definition of hypothesis discovery: Generation (creating hypotheses that explain given observations with abduction), Application (deducing new observations from established hypotheses with deduction), Validation (verifying and refining hypotheses against new evidence with induction), and Integrated Hypothesis Discovery (examining the dynamic interdependencies among these components in a continuous, iterative process). Hypothesis discovery inherently relies on a blend of reasoning that includes abduction, induction, and deduction, each defined differently by various scholars. For instance, Gilbert H. Harman considers induction to be a special case of abduction, describing it as “inference to the best explanation” (IBE) ( Harman, 1965 ; Douven, 2021 ) . However, while this definition is easy to understand, it oversimplifies key aspects of hypothesis discovery. In particular, the notion of the “best” explanation is ambiguous and often requires additional assumptions that vary by context. Moreover, this framework does not fully capture real-world scenarios, where a “best” explanation is rarely reached immediately; rather, we continually experiment, gather new observations, and refine our hypotheses. Based on these considerations, we adopt Charles Peirce’s definition of hypothesis discovery and reasoning, which posits that hypothesis discovery begins with forming an explanatory hypothesis to explain observations through abduction , proceeds with iteratively apply hypothesis to solve problem or derive new knowledge with deduction , and validate hypothesis through induction ( Frankfurt, 1958 ; Peirce, 1974 ; Burks, 1946 ; Minnameier, 2004 ) (See explanation in Figure 2 ). The rest of the survey is organized as follows. Section 2 presents background knowledge on hypothesis discovery using LLMs, including different forms of reasoning and representations involved in the process. Section 3 examines prior surveys on LLM reasoning and hypothesis discovery, highlighting their narrow emphasis on deductive tasks or application-specific methods. Section 4 reviews methods for forming hypotheses (Abduction) . Section 5 then covers approaches for applying these hypotheses (Deduction) , and Section 6 focuses on techniques for validating given hypotheses with new observations (Induction) . Finally, Section 7 explores the entire hypothesis-discovery cycle by examining the interdependencies among these reasoning steps and showing how abduction, deduction, and induction can be iteratively used to refine more robust hypotheses. For each stage, we discuss methods, benchmarks, evaluations, and identify limitations and future directions. A high-level taxonomy guiding this survey is shown in Figure 1 . Figure 2: On the left-hand side, a) illustrates Peirce’s framework for hypothesis discovery through abduction, deduction, and induction. The process begins with abduction, which generates explanatory hypotheses based on an initial set of observations. Deduction is then used to apply these hypotheses and derive predictions. Induction evaluates how well the predicted observations align with actual outcomes, updating the confidence of the hypotheses or rejecting those that are no longer valid. This process is iterative: validated hypotheses may be refined through further rounds of abduction using updated observations, gradually leading to more robust theories. On the right-hand side, b) provides a simple example that illustrates this process. 2 Background Before LLMs, most AI systems stored knowledge as handcrafted symbols and rules. That format works well for deduction, because most of the problems we need to solve with symbolic AI systems work with limited premises and countable task-specific knowledge; for example, questions in the ProofWriter ( Tafjord et al., 2021 ) and FOLIO ( Han et al., 2024 ) benchmarks are limited to fewer than a hundred premises. However, abduction and induction are different: they call for generating and validating many tentative explanations inspired by vast commonsense or expert domain knowledge (such as weather patterns, social norms, or physics) and for updating beliefs as new observations arrive. Handling these reasoning tasks with symbolic AI meant writing and maintaining vast, interlocking rule bases, an effort so costly that few projects moved beyond toy domains ( Yang et al., 2024c ) . Consequently, the research landscape remained dominated by deductive tasks ( Yu et al., 2024a ; Liu et al., 2025 ; Huang & Chang, 2023 ) . LLMs have transformed this landscape. Trained on vast corpora, they implicitly absorb broad commonsense and domain knowledge, exhibiting strong reasoning abilities on complex, natural-language tasks ( Yang et al., 2024c ) . With a simple text prompt, we can now ask them to carry out abduction or induction and even inspect their intermediate reasoning steps ( Li et al., 2024 ; Jung et al., 2022 ) , exposing the latent information they rely on. This advancement has made it practical to study and deploy defeasible reasoning ( Yang et al., 2024c ) . Defeasible reasoning refers to forms of reasoning, such as abduction and induction, that yield probable conclusions that remain open to revision as new evidence emerges. This shift has fueled a wave of NLP research that places such flexible reasoning at the heart of AI progress ( Liu et al., 2025 ; Huang & Chang, 2023 ) . 2.1 Hypothesis Discovery Hypothesis discovery or Rule learning, the cyclical process of formulating hypotheses, gathering evidence, validating or refuting them, and ultimately establishing robust theories, lies at the heart of scientific progress ( Eger et al., 2025 ) . Early humans, for example, hypothesized that the Earth was flat based on everyday observations. Later, Eratosthenes measured shadow angles at different locations, obtaining evidence that suggested the Earth’s surface was curved. This evidence challenged the flat Earth hypothesis, and subsequent findings, notably Magellan’s circumnavigation, conclusively confirmed the Earth’s roundness. Even with today’s sophisticated instruments, researchers continue to iterate this loop in new domains, validating and refining theories as new data emerges. Today, there is growing interest in whether LLMs can autonomously generate, apply, and

Read the complete text