Probabilistic Attribution For Large Language Models
Shilpika [email protected], Carlo Graziani [email protected], Bethany Lusch [email protected], Venkatram Vishwanath [email protected], Michael E. Papka [email protected] · 2026
Read Probabilistic Attribution For Large Language Models by Shilpika [email protected], Carlo Graziani [email protected], Bethany Lusch [email protected], Venkatram Vishwanath [email protected], Michael E. Papka [email protected] (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-21726.txt.
Introduction
Shilpika and colleagues recast language models as stochastic processes, using the next-token probabilities a model assigns at each step. By inverting those probabilities with Bayes rule, they build a new attribution score. The score is the log ratio of response probability with a prompt token present versus marginalized away. They also compute token entropies given the remaining context. Across eight models and seven prompts, the interplay of entropy and attribution exposes unstable or uncertain parts of a generation.
Catalog record
| CREATOR | Shilpika [email protected], Carlo Graziani [email protected], Bethany Lusch [email protected], Venkatram Vishwanath [email protected], Michael E. Papka [email protected] |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 61279 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2605.21726
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Probabilistic Attribution For Large Language Models Shilpika [email protected], Carlo Graziani [email protected], Bethany Lusch [email protected], Venkatram Vishwanath [email protected], Michael E. Papka [email protected] Abstract The generative nature of Large Language Models (LLMs) is reflected in the conditional probabilities they compute to sample each response token given the previous tokens. These probabilities encode the distributional structure that the model learns in training and exploits in inference. In this work, we use these probabilities to situate LLMs within the mathematical theory of stochastic processes. We use this framework to design a model-agnostic probabilistic token attribution measure, using Bayes rule to invert the next-token log-probabilities so as to capture the model’s internal representation of the distribution over token sequences. The representation is independent of the model’s computational structure. This representation yields the conditional probability of the response given the prompt, and of the response given the prompt with a token marginalized away. Our attribution score is the log of the ratio of these probabilities. We further compute the entropies of a single prompt’s token distributions, conditioned on the remaining context. The interplay between entropy and attribution score sheds light on LLM behavior. We evaluate 8 models across 7 prompts and investigate anomalies, token sensitivity, response stability, model stability, and training convergence, thereby improving interpretability and guiding users to focus on uncertain or unstable parts of the generation. Author One and Author Two † † heading: 23 2022 1- 1/21; Revised 5/22 9/22 21-0000 † † shortheadings: Probabilistic Attribution For Large Language Models / One and Two † † firstpage: 1 † † editor: Pradeep Ravikumar, Carnegie Mellon University, Tong Zhang, University of Illinois Urbana-Champaign. keywords AI, explainability, explainable AI, XAI, interpretability, entropy, probability theory 1 Introduction The growing impact of AI on scientific modeling mandates a newly-rigorous approach to analyzing what AI models do, how they do it, and what uncertainties should be associated with AI-based predictions and decisions. Key concerns include appropriate model sizing (which directly impacts the energy cost of training), output validity/correctness, and output uncertainty, among others. These present substantial challenges, because the complexity of the models precludes the kind of direct interpretability that attends first-principles mathematical/statistical models, making it necessary to address such questions indirectly. To enable the understanding of the decision-making process of AI models, particularly large language models, several techniques, frameworks, and methodologies have been developed, and these are referred to as Explainable Artificial Intelligence (XAI). XAI provides transparency into how inputs to “black box” AI model generate a response. The motivation for XAI arises from the need to foster trust, regulation, accountability, fairness, and safety in AI applications, especially in high-stakes domains like healthcare, finance, and law, where opaque decision-making can have serious ethical and social consequences ( 15 ; 20 ) . XAI encompasses methods that can be broadly categorized into model-specific approaches, which are tailored to certain classes of models (for example, saliency maps and gradient-based visualizations in deep neural networks), and model-agnostic approaches, which can be applied universally regardless of the underlying algorithm ( 1 ) . Model-agnostic methods such as SHAP (SHapley Additive exPlanations) and LIME (Local Interpretable Model-agnostic Explanations) rely on input–output behavior rather than internal architecture, making them particularly versatile across different machine learning models ( 12 ; 18 ) . In this work, we situate the inference operations of large language models (LLMs) within the mathematical theory of stochastic processes. Using this framework, we develop a probabilistic attribution score ( AS ) for large-language models that is model-agnostic. The AS determines the importance of tokens in a prompt given a generated response. LLMs are fundamentally autoregressive, meaning they generate text one token at a time by sampling a computed probability distribution for the next output token conditioned on all the previous ones. LLMs work because they successfully infer statistical properties of token sequences from training corpuses and exploit those properties to make near-optimal prompt-response decisions. This process is directly grounded in the chain rule of probability, which allows the joint probability of a sequence of random variables to be factorized into a product of conditional probabilities. Stochastic process theory provides a useful mathematical framework for understanding those operations in a model-agnostic fashion that permits easy comparison between different types of LLMs. This framework also permits useful probabilistic analyses of LLM responses, of which AS is an example. Our work has the following contributions: C1. AS is model-agnostic and is applied to text-generation tasks, although it can be extended to any NLP-based task. C2. Our method is parameter-free, which makes it less sensitive to arbitrary user choices and improves reproducibility and objectivity, making it suitable for hyperparameter-tuning of other XAI methods. C3. Our contextual and replacement entropy are measures of model stability, and can indicate how well these models converged to their training data distribution. C4. AS has built-in sensitivity, as it is based on statistical first principles and thus evaluates the correctness of the explainable results as part of its core methodology. The remainder of this paper is organized as follows. The Background section reviews the key concepts, prior work, and theoretical foundations relevant to the study. The LLMs as Stochastic Processes, Experiments, and Evaluation sections then describe the proposed approach, experimental setup, implementation details, comparative analysis, and the criteria used to assess performance. Finally, the Conclusion summarizes the main findings, discusses their implications, and outlines potential directions for future research. The code is available at: https://github.com/sshilpika/probabilistic-attribution-score/ . 2 Background: Explainable AI for NLP and Large Language Models Explainable artificial intelligence (XAI) in natural language processing (NLP) has evolved from post hoc interpretability methods for linear models and early neural architectures into a diverse methodological landscape tailored to Transformer encoders and contemporary large language models ( 13 ; 19 ) . Current research spans three broad levels: local explanations of individual predictions, global analyses of model behavior and internalized knowledge, and mechanistic interpretability aimed at identifying the internal computational structures that give rise to model outputs ( 13 ; 6 ) . Early work in NLP XAI focused on local feature-attribution methods designed to estimate which input tokens or n-grams most influenced a model’s prediction. LIME represented a major advance by explaining predictions through locally fitted interpretable surrogate models, often using perturbation-based token deletion ( 18 ) . Closely related Shapley-value-based methods, particularly SHAP, provided a more principled additive framework for attribution and became widely adopted in text-based interpretability pipelines ( 12 ) . For neural NLP models, gradient-based methods such as Integrated Gradients became especially influential due to their computational scalability and axiomatic grounding ( 24 ) . Toolkits such as Captum further standardized the implementation and comparison of attribution methods in PyTorch-based workflows ( 10 ) . However, it soon became clear that token-level attributions could be unstable under paraphrase, and sensitive to perturbation strategy ( 23 ) . These limitations motivated the development of rationale-based explanations, which identify spans of text intended to justify a prediction. This line of work was consolidated by benchmarks such as ERASER, which distinguished plausibility, or agreement with human-annotated rationales, from faithfulness, or the extent to which a rationale genuinely supports the model’s prediction ( 3 ) . This distinction has become foundational in LLM explainability, where persuasive explanations may still fail to