LLM Cannot Discover Causality, and Should Be Restricted to Non-Decisional Support in Causal Discovery
Xingyu Wu, Kui Yu, Jibin Wu, Kay Chen Tan · 2025
Read LLM Cannot Discover Causality, and Should Be Restricted to Non-Decisional Support in Causal Discovery by Xingyu Wu, Kui Yu, Jibin Wu, Kay Chen Tan (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2506-00844.txt.
Introduction
Xingyu Wu and co-authors argue that large language models should be barred from deciding causal claims, however fluent their reasoning sounds. Causal discovery is booming on model assistance, with systems asking chatbots which variable causes which. Their 2025 paper contends the practice is built on sand: an autoregressive model trained on correlations has no machinery for causal direction.
Empirical probes expose the failure modes, and the authors show that favorable results in the literature often trace to prompts quietly fed the true answer. The permitted role is humble but real — the model may suggest search heuristics for causal graph algorithms, never vote on edges. Guardrailed that way, its help stops hurting. Causal inference researchers get a boundary stone with experiments behind it.
Catalog record
| CREATOR | Xingyu Wu, Kui Yu, Jibin Wu, Kay Chen Tan |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 78 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2506-00844
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
LLM Cannot Discover Causality, and Should Be Restricted to Non-Decisional Support in Causal Discovery Xingyu Wu, Kui Yu, Jibin Wu, Kay Chen Tan Abstract This paper critically re-evaluates LLMs’ role in causal discovery and argues against their direct involvement in determining causal relationships. We demonstrate that LLMs’ autoregressive, correlation-driven modeling inherently lacks the theoretical grounding for causal reasoning and introduces unreliability when used as priors in causal discovery algorithms. Through empirical studies, we expose the limitations of existing LLM-based methods and reveal that deliberate prompt engineering (e.g., injecting ground-truth knowledge) could overstate their performance, helping to explain the consistently favorable results reported in much of the current literature. Based on these findings, we strictly confined LLMs’ role to a non-decisional auxiliary capacity: LLMs should not participate in determining the existence or directionality of causal relationships, but can assist the search process for causal graphs (e.g., LLM-based heuristic search). Experiments across various settings confirm that, by strictly isolating LLMs from causal decision-making, LLM-guided heuristic search can accelerate the convergence and outperform both traditional and LLM-based methods in causal structure learning. We conclude with a call for the community to shift focus from naively applying LLMs to developing specialized models and training method that respect the core principles of causal discovery. 1 Introduction Causal discovery aims to uncover underlying causal relationships from observational data [ 1 ] . Traditional causal discovery algorithms (CDAs) construct causal structures by systematically leveraging conditional independence tests, optimization techniques, or functional causal models. These methods integrate theoretical principles from graphical models, information theory, and structural causal modeling to ensure interpretability and maintain theoretical guarantees in causal discovery [ 2 ] . Recently, large language models (LLMs) have emerged as a potential alternative for causal discovery [ 3 ] . Many studies have explored the application of LLMs in this domain, leveraging their linguistic knowledge and common-sense reasoning to identify causal relationships. These efforts have sparked significant interest, particularly in investigating whether LLMs can assist in identifying causality from text and structured data. LLMs have achieved notable performance in causal discovery, particularly in identifying pairwise causal relationships from text [ 4 , 5 ] . Existing studies have leveraged variable information [ 6 ] and background knowledge [ 7 ] to enhance LLMs’ ability to identify causality. These studies not only highlight LLMs’ strengths in recognizing pairwise causality but also explore hybrid approaches that integrate LLMs with traditional CDAs to enhance performance [ 8 ] . For instance, researchers have utilized LLMs to initialize causal structures [ 9 ] or refine CDA-generated graphs [ 10 ] . Most studies demonstrate promising results, suggesting that LLMs hold significant potential in causal discovery. However, these efforts primarily focus on empirical evaluations, leaving open questions about whether LLMs truly possess causal reasoning abilities. Some researchers have raised concerns that LLMs may struggle with complex causal tasks [ 7 ] and are prone to generating hallucinated causal relations, i.e., assigning causality where none exists [ 11 ] . Given the instability of LLMs, several mitigation strategies have been proposed. These include providing observational data [ 12 ] , employing LLMs as conditional independence test operators [ 13 ] , and incorporating LLM-generated causal judgments into classical scoring functions as priors [ 5 ] . While these improvements claim to enhance reliability to some extent, they do not fully resolve the fundamental limitations of LLMs in causal discovery. Figure 1: This logical framework illustrates the focus and main positions of Sections 2 to 6. Building on Section 2’s comprehensive review of existing methods, Sections 3 and 4 conduct a thorough critique of current algorithms, whether using LLMs as hard knowledge or soft knowledge. Then, Section 5 defines the scope of LLMs’ role in causal discovery. Section 6 further reveals the manipulability of experimental results in existing methods, which explains why they achieve good performance. Different from most existing studies, we argue that the role of LLMs in causal discovery tasks is limited. Through the following critical and compensatory perspectives, we clearly define the current scope of LLMs’ applicable functions: 1. Critical Position: LLMs cannot identify causality, and their outputs should not directly or indirectly determine the existence or directionality of causal relationships. This includes: • Direct Result Determination : LLMs should not independently conclude whether a causal relationship exists ( Section 3 ). • Indirect Influence on Results : Causal relationships identified by LLMs should not be embedded as "prior knowledge", whether as hard constraints or soft penalties, into the decision-making processes of CDAs ( Section 4 ). 2. Compensatory Position: Without influencing causal decision-making, LLMs can assist CDAs by improving the search process for causal graphs, thereby accelerating the convergence toward optimal causal structures ( Section 5 ). To this end, this paper investigates the limitations of LLMs in causal discovery from two critical perspectives: (i) the intrinsic limitations of LLMs to infer causality, and (ii) the challenges arising from their integration with classical CDAs. As shown in Figure 1 , Section 3 first explains how LLMs’ autoregressive modeling based on word order [ 14 ] inherently differs from the probability decomposition in structural causal models [ 15 ] , leading to a lack of theoretical grounding for causal reasoning. Through a series of empirical studies, we systematically analyze the impact of several key factors—such as word order, redundancy, entity positioning, and numerical precision—on LLMs’ causal inference. Results reveal that LLM performance is highly sensitive to prompt design and text quality, and merely supplying observational data does not guarantee improved reliability. Unless causal relationships are stated with exceptional clarity and minimal noise, LLMs often fail to reach consistent or correct conclusions. Given these reliability concerns, Section 4 further investigates the interaction between LLMs and traditional CDAs, analyzing different collaboration paradigms and their limitations. We focus on how using LLM-derived causal relationships as (hard/soft) prior knowledge in CDAs [ 5 ] compromises algorithmic reliability and theoretical guarantees, regardless of whether the method is score-based or constraint-based. Moreover, in Section 6 , we demonstrate that the performance of LLM-based methods can be artificially inflated via prompt engineering, for instance, by injecting ground-truth into the input. Building on these analyses, we advocate for a principled delineation of LLMs’ functional boundaries in causal discovery: LLMs should not determine the existence or directionality of causal relationships, but can serve as auxiliary agents that guide the causal graph search process. Section 5 introduces a novel paradigm where LLMs assist in heuristic search—such as through initialization, mutation guidance, or cycle resolution—while preserving the decision-making authority of theoretically sound CDAs. By leveraging LLMs’ general world knowledge [ 16 ] in a non-decisional role, this framework helps CDAs escape local optima and converge faster to globally optimal structures. Crucially, the evaluation of causal structures remains grounded in statistical analysis of observational data via established scoring functions or independence tests, which is fundamentally different from existing LLM-based causal discovery methods. Comprehensive experiments across varying network scales confirm that when LLMs are appropriately confined to assistive roles, they can significantly enhance search efficiency and accuracy compared to both traditional CDAs and existing LLM-based methods. Thus, our results underscore the value of LLMs as heuristic facilitators—not causal reasoners—in the pursuit of reliable causal discovery.