Quantifying Symptom Causality in Clinical Decision Making: An Exploration Using CausaLM
Connor Jordan, Mehul Shetty · 2025
Read Quantifying Symptom Causality in Clinical Decision Making: An Exploration Using CausaLM by Connor Jordan, Mehul Shetty (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2503-19394.txt.
Introduction
Alexander Romanus and co-authors probe how much a single symptom truly causes a diagnostic model's prediction, in a 2025 paper. Most medical models learn correlations between symptoms and diseases. Ambiguous symptoms can then mislead them. Chest pain, for example, appears in many conditions.
The team uses the CausaLM framework to build counterfactual text. In that text, a target concept is made to be forgotten by the model. A measure called TReATE then estimates the symptom's causal effect on predictions. Comparing outcomes with and without the symptom shows its real weight. The approach moves diagnosis support beyond pattern matching. It could help doctors trust, or question, model outputs with better reason.
Catalog record
| CREATOR | Connor Jordan, Mehul Shetty |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 21 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2503-19394
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Quantifying Symptom Causality in Clinical Decision Making: An Exploration Using CausaLM Connor Jordan, Mehul Shetty Abstract Current machine learning approaches to medical diagnosis often rely on correlational patterns between symptoms and diseases, risking misdiagnoses when symptoms are ambiguous or common across multiple conditions. In this work, we move beyond correlation to investigate the causal influence of key symptoms—specifically “chest pain”—on diagnostic predictions. Leveraging the CausaLM framework, we generate counterfactual text representations in which target concepts are effectively “forgotten,” enabling a principled estimation of the causal effect of that concept on a model’s predicted disease distribution. By employing Textual Representation-based Average Treatment Effect (TReATE), we quantify how the presence or absence of a symptom shapes the model’s diagnostic outcomes, and contrast these findings against correlation-based baselines such as CONEXP. Our results offer deeper insight into the decision-making behavior of clinical NLP models and have the potential to inform more trustworthy, interpretable, and causally-grounded decision support tools in medical practice. 1 Introduction One of the key issues in causal inference in NLP is the generation of counterfactuals. To generate counterfactuals we need a controlled setting where it is possible to compute the differences between actual text and what the text would have been if a specific concept in the text had not existed. Here, a concept refers to an entire semantic space. While there are many methods to generate counterfactuals manually, they are often arduous and impossible to generate for large datasets. It is much easier to instead generate a counterfactual text representation based on adversarial learning than to create counterfactual text. We plan to use the CausaLM ( Feder et al. 2021 ) framework to create counterfactual text representations for clinical notes that "forget" a target concept while making sure it does not forget other potential confounders called control concepts. This innovative approach leverages the power of causal language models to uncover the intricate relationships between symptoms and diagnoses in medical decision-making. Specifically, we will use medical reports describing patient symptoms that is ingestible by CausaLM through tokenization. Once the data is processed in a structured format, it will be optimal for use in the CausaLM model. We will then use it to predict the counterfactual diagnoses as if certain symptoms had not been considered by the doctor—allowing us to explore alternative diagnostic pathways, and the weights given to various symptoms in the diagnostic process. By implementing this approach, we can simulate situations where these symptoms were either not observed or not reported, providing insight into how the absence of key symptoms might alter the diagnostic outcome. We hope this will give us an accurate measure of the causal effect that these symptoms have on a specialist referral, creating a more robust and systematic way to compare diagnoses. This research could have far-reaching implications for improving clinical decision support systems, enhancing medical education, and ultimately improving patient outcomes through more precise and personalized diagnostic approaches. 2 Related Work Making inferences about patients’ diagnoses based on symptoms is commonly seen in many NLP models in medical contexts. Research has focused on finding the relationship between clinical notes and the doctors’ resulting diagnosis using LLM. Mullenbach et al. 2018 show NLP models making predictions for resulting ICD codes from patient data. Xie et al. 2024 combine both text and images as input to provide medical queries using LLMs. Michalopoulos et al. 2021 combine the uses of common NLPs with their own context embedding system to understand and make accurate predictions based on large amounts of clinical text. While these models make accurate diagnoses, none of them apply causality to their predictions as we proposed. Causal machine learning concepts have also been used in the medical setting to improve patient outcomes, such as Richens et al. 2020 where causal machine learning is used as a way to improve the accuracy of the various algorithms and frameworks in the model to properly diagnoses rare diseases. The LLM proposed in this project differs with its use of causal reasoning compared to the causal machine learning concepts used by Richens et al. 2020 . Gopalakrishnan et al. 2024 used automated techniques to identify causalities from an annotated set of medical data. The researchers in this study utilize BioBERT to perform causal extraction tasks, enabling the collection of large datasets to enhance clinical decision-making and patient care. Ganin et al. 2016 utilizes a technique called a "gradient reversal layer" to adversarially learn domain-invariant features. In their approach, they were able to successfully improve performance on various tasks such as sentiment analysis and image classification through domain invariance. In our work, we can use the concept of the gradient reversal layer to help our model "forget" a symptom and in turn adversarially train it and measure the impact of different symptoms. Perhaps the work that we derive most heavily from is Feder et al. 2021 . We will be using the underling framework proposed in this paper but finetuning it specifically for clinical decision making. 3 Methodology To describe our methodology we first need to define causal model explanation. While causal inference is the main objective in many scientific endeavors, we rely here on a completely different aspect called causal model explanation. We’re trying to find the causal effect of a given concept (called treatment) on the model’s predictions, and show these effects to explain the observed behavior of the model. Here, a concept refers to a higher level, often aggregated unit, atomic input features such as words. It’s an abstract idea rather that a collection of words. 3.1 Language Representation Our approach is based on the idea that any text is created by a number of concepts coming together through a data-generating process ( Feder et al. 2021 ) . Figure 1 describes this process. Imagine that we observe a clinical note X and have trained a model to give a disease distribution based on the symptoms. We can hypothesize a list of concepts that might affect the model’s decisions. We will denote the set of binary variables C=\{C_{j}\in\{0,1\}|j\in\{0,1,2,...,k\} , where each variable corresponds to the existence of a predefined concept (in our case each symptom). If C_{j}=1 that means the j-th symptom exists in the text. We assume a pretrained language model \phi and wish to assert how our trained disease distribution model f is affected by the concepts in C . Figure 1: Three causal graphs relating the concepts of symptoms, clinical notes, their representations and distribution output. The top graph describes the original data-generating process g . The middle graph describes the case of directly manipulating the text using an alternative generative process g^{C1} that allows us to create a text representation X^{\prime} that is the same as X but does not contain the concept C_{1} . The bottom graph describes our approach where we manipulate the text representation and not the text itself. The dashed edges between concepts indicate possible hidden confounders. 3.2 Representation-Based Counterfactual Generation Next we will discuss how went about creating our model using the CausaLM framework. We first used a pre-trained BERT and finetuned it for our goal. As described in Figure 2, we used the original BERT NSP and MLM heads and added a new TC head that had