Debiasing Large Language Models via Adaptive Causal Prompting with Sketch-of-Thought
Bowen Li, Ziqi Xu, Jing Ren, Renqiang Luo, Xikun Zhang, Xiuzhen Zhang, Yongli Ren, Feng Xia · 2026
Read Debiasing Large Language Models via Adaptive Causal Prompting with Sketch-of-Thought by Bowen Li, Ziqi Xu, Jing Ren, Renqiang Luo, Xikun Zhang, Xiuzhen Zhang, Yongli Ren, Feng Xia (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-08108.txt.
Introduction
Bowen Li and seven co-authors present a prompting method that aims to cut bias in language models. Their 2026 arXiv paper notes that methods like chain-of-thought have improved reasoning. Yet many prompting strategies use too many tokens and generalize poorly across varied reasoning tasks.
The team combines adaptive causal prompting with sketch-of-thought, a compact reasoning style. Causal prompting targets the sources of biased answers rather than surface wording alone. Sketch-of-thought keeps the reasoning trace short, which lowers token cost. For practitioners, the paper offers a route to fairer answers that is also cheaper to run across diverse tasks.
Catalog record
| CREATOR | Bowen Li, Ziqi Xu, Jing Ren, Renqiang Luo, Xikun Zhang, Xiuzhen Zhang, Yongli Ren, Feng Xia |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 64 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2601-08108
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Debiasing Large Language Models via Adaptive Causal Prompting with Sketch-of-Thought Bowen Li, Ziqi Xu, Jing Ren, Renqiang Luo, Xikun Zhang, Xiuzhen Zhang, Yongli Ren, Feng Xia Abstract Despite notable advancements in prompting methods for Large Language Models (LLMs), such as Chain-of-Thought (CoT), existing strategies still suffer from excessive token usage and limited generalisability across diverse reasoning tasks. To address these limitations, we propose an A daptive C ausal P rompting with S ketch-of-Thought ( ACPS ) framework, which leverages structural causal models to infer the causal effect of a query on its answer and adaptively select an appropriate intervention (i.e., standard front-door and conditional front-door adjustments). This design enables generalisable causal reasoning across heterogeneous tasks without task-specific retraining. By replacing verbose CoT with concise Sketch-of-Thought, ACPS enables efficient reasoning that significantly reduces token usage and inference cost. Extensive experiments on multiple reasoning benchmarks and LLMs demonstrate that ACPS consistently outperforms existing prompting baselines in terms of accuracy, robustness, and computational efficiency. The source code can be found at https://aisuko.github.io/acps/ . 1 Introduction Large Language Models (LLMs) play a central role in Natural Language Processing (NLP), achieving state-of-the-art results across a wide range of tasks, from open-domain question answering to multi-step logical reasoning 2 . Building on this success, prompt-based methods extend LLM capabilities without requiring full model retraining. For example, In-Context Learning (ICL) 31 introduces example demonstrations directly into the prompt, enabling LLMs to generalise from just a few instances. To support more complex reasoning, Chain-of-Thought (CoT) prompting elicits step-by-step inference, substantially improving performance on multi-hop and logical tasks 29 . Despite recent advances, current prompting strategies exhibit two critical shortcomings. First, excessive token generation remains a major issue: CoT prompts often produce unnecessarily lengthy or redundant reasoning chains, with hundreds of tokens per query, which reduces efficiency and increases inference cost 33 . While methods such as Chain-of-Draft (CoD) 33 and Sketch-of-Thought (SoT) 1 attempt to alleviate this, they typically lag behind CoT in accuracy. Second, unfaithful reasoning emerges from internal model biases, where LLMs may rationalise a predisposed answer instead of performing genuine inference. Subtle bias cues in prompts can lead to plausible yet factually incorrect CoTs, resulting in accuracy drops of up to 36% on complex reasoning tasks 27 . Such reasoning failures undermine trust in applications that require faithful and verifiable explanations, such as scientific question answering and commonsense inference. Recent research has explored the integration of causal inference with LLMs as a promising direction for addressing the aforementioned challenges. By incorporating causal principles such as standard front-door adjustment and instrumental variable, it becomes possible to mitigate bias caused by unobserved confounders, which is often interpreted as internal bias in LLMs. These unobserved confounders can introduce spurious correlations between the query and the answer, leading to unfaithful or misleading outputs. For example, Causal Prompting 37 estimates the causal effect of a query its answer by controlling for intermediate reasoning steps such as CoT. Instead of relying on majority voting across multiple reasoning paths, this method selects the answer with the highest estimated causal effect, thereby improving both accuracy and interpretability. However, despite its potential, existing causality-based prompting methods rely on strong assumptions, including the identifiability conditions required by front-door adjustment and the validity of instrumental variables. These assumptions may not hold consistently across different NLP tasks, limiting the generalisability of such methods in practice. Furthermore, the reliance on CoT often leads to verbose outputs, which increase both token usage and inference cost. Thus, there is a pressing need for a causality-based prompting framework that mitigates internal biases in LLM reasoning by adaptively selecting an appropriate intervention for different NLP tasks, while efficiently reducing token usage without compromising performance. Figure 1: An example from GPT-3.5-turbo on the GSM8K dataset. Left: (i) Recent non-causal prompting methods often amplify internal bias through majority voting. (ii) Some causality-based prompting methods mitigate this bias but rely on verbose CoT, leading to high token usage and inference cost. Right: (iii) The proposed framework uses SoT instead of CoT and selects the answer based on the highest estimated causal effect, yielding the correct result. In this work, we propose an A daptive C ausal P rompting with S ketch-of-Thought ( ACPS ) framework to enhance both the generalisability and efficiency of debiasing LLMs. ACPS adaptively applies standard or conditional front-door adjustment depending on the characteristics of each NLP task, supported by a classification engine. To improve inference efficiency, it replaces verbose CoT with concise SoT, significantly reducing token usage and computational cost. As shown in Figure 1 , ACPS produces the correct answer on a representative GSM8K example. In contrast, non-causal prompting methods suffer from internal bias, while causality-based prompting methods that rely on verbose CoT are constrained by inefficiency. This highlights how our adaptive formulation and concise reasoning traces contribute to improved accuracy and efficiency in inference. The main contributions of this paper are as follows: • We propose a novel framework, ACPS, for debiasing LLMs via adaptive causal prompting. This model-agnostic mechanism selects an appropriate intervention based on task characteristics, thereby overcoming the limitations of fixed prompting and improving generalisability across diverse reasoning tasks. • To the best of our knowledge, ACPS is the first framework that integrates both standard and conditional front-door adjustments with SoT. This integration significantly reduces token usage and inference cost, while preserving high reasoning accuracy. • We validate our framework through extensive experiments across multiple LLMs and reasoning benchmarks, demonstrating consistent improvements in accuracy, efficiency, and robustness over existing prompting baselines. 2 Preliminaries In this section, we review the fundamental concepts of causality and sketch-of-thought. 2.1 Structural Causal Model We adopt the structural causal model (SCM) 17 , in which each endogenous variable is determined by a deterministic function of its parent variables and an independent exogenous noise term. The causal structure is represented by a directed acyclic graph (DAG), where nodes correspond to random variables and directed edges denote direct causal relationships. Exogenous variables are assumed to be mutually independent, allowing the joint distribution to factorise according to the graph structure. We assume the Markov condition ( 18 ) , which states that each variable is conditionally independent of its non-effects given its direct causes, and the faithfulness assumption ( 24 ) , which asserts that all and only those conditional independencies implied by the DAG are present in the observed distribution. Under these assumptions, d-separation ( 18 ) can be used as a criterion to determine conditional independence relationships. Interventions, denoted by do(X=x) , modify the structural equation for X , producing a new distribution that reflects the causal effect of the intervention. For formal definition and further details, see ( 18 ) . For direct prompt-to-answer reasoning, the conceptual-level SCM is illustrated in Figure 2(a) . The query Q , which includes both demonstrations and test examples provided to the LLM, leads to the answer A . Although the direct causal effect from Q to A is represented as Q\to A , LLMs often internalise biases from large-scale pre-training corpora 6 ; 38 ; 20 . To account for these biases, we introduce an unobserved variable U that influences both Q and A . The presence of U creates a spurious association between