Robust Explanations for User Trust in Enterprise NLP Systems

Guilin Zhang, Kai Zhao, Jeffrey Friedman · 2026

Read Robust Explanations for User Trust in Enterprise NLP Systems by Guilin Zhang, Kai Zhao, Jeffrey Friedman (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-12069.txt.

Introduction

Zhang and five colleagues propose a black-box evaluation for token-level explanations in language systems, where API-only access blocks explainers. They measure top-token flip rate under perturbations — swapping, deletion, shuffling, and back-translation. Across three datasets and six encoder and decoder models, with 64,800 cases, decoders prove more stable than encoder baselines, with 73 percent lower flip rates. Stability improves with scale, gaining 44 percent from 7B to 70B. Relating scores to inference cost yields a cost tradeoff curve for predeployment model selection.

Catalog record

CREATORGuilin Zhang, Kai Zhao, Jeffrey Friedman
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE28526
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2604.12069

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Robust Explanations for User Trust in Enterprise NLP Systems Guilin Zhang, Kai Zhao, Jeffrey Friedman Abstract Robust explanations are increasingly required for user trust in enterprise NLP, yet pre-deployment validation is difficult in the common case of black-box deployment (API-only access) where representation-based explainers are infeasible and existing studies provide limited guidance on whether explanations remain stable under real user noise, especially when organizations migrate from encoder classifiers to decoder LLMs. To close this gap, we propose a unified black-box robustness evaluation framework for token-level explanations based on leave-one-out occlusion, and operationalize explanation robustness with top-token flip rate under realistic perturbations (swap, deletion, shuffling, and back-translation) at multiple severity levels. Using this protocol, we conduct a systematic cross-architecture comparison across three benchmark datasets and six models spanning encoder and decoder families (BERT, RoBERTa, Qwen 7B/14B, Llama 8B/70B; 64,800 cases). We find that, at the model sizes we evaluate, decoder LLMs produce substantially more stable explanations than encoder baselines (73% lower flip rates on average), with stability improving further within the decoder family with scale (44% gain from 7B to 70B). Finally, we relate robustness improvements to inference cost, yielding a practical cost–robustness tradeoff curve that supports model and explanation selection prior to deployment in compliance-sensitive applications. 1 Introduction Robust explanations are increasingly required for user trust in enterprise NLP systems, especially in finance and HR platforms where text classifiers support high-impact decisions (e.g., risk screening, policy compliance, case triage, and ticket routing). Meanwhile, many organizations are migrating from encoder-based models such as BERT 6 and RoBERTa 13 to decoder-based LLMs such as GPT 5 , Llama 22 ; 10 , and Qwen 4 ; 25 due to improved accuracy and flexibility, particularly in few-shot and zero-shot settings 24 . However, beyond accuracy, it remains unclear whether this architectural shift improves the robustness of explanations needed for enterprise deployment. This gap is amplified by a common production reality: enterprise models are often accessed through governed services or third-party APIs, exposing only inputs and outputs and thus imposing black-box constraints that preclude gradients or internal representations. Under such constraints, teams depend on query-based, model-agnostic explainers and must ensure that explanations are stable under routine user-like noise (edits, rephrasing, reordering). Unstable explanations undermine auditability and user trust and can complicate compliance obligations, including emerging requirements for consistent and traceable rationales. These considerations lead to a practical pre-deployment question: under black-box access and realistic input perturbations, do decoder LLMs provide more stable explanations than encoder models, and what tradeoffs arise in cost and latency? Prior work has proposed a broad set of NLP explanation methods, including attention visualization 23 , gradient-based attribution 21 , and perturbation-based techniques such as LIME 19 and SHAP 14 . Robustness analyses, however, have largely focused on single architectures 2 ; 15 or compared explainers rather than model families 3 ; 8 . Separately, work on adversarial text perturbations 11 ; 18 and robustness of LLM behavior 1 ; 27 demonstrates sensitivity to input variation, but cross-architectural explanation stability —evaluated under a unified, black-box protocol and mapped to deployment tradeoffs—remains insufficiently characterized. To address this gap, we designed a pre-deployment study that mirrors how explainability is evaluated in enterprise NLP systems when models are only accessible through governed endpoints or third-party APIs. We built a unified black-box evaluation framework for token-level explanations using leave-one-out occlusion, and we operationalized robustness as explanation stability under realistic user-like perturbations, including swap, deletion, shuffling, and back-translation at multiple severity levels. We then applied this framework consistently across six representative models spanning encoder and decoder families (BERT, RoBERTa, Qwen 7B/14B, Llama 8B/70B) on three text classification benchmarks, generating 64,800 paired evaluations under matched perturbations and identical explanation procedures. From this controlled setup, we observed a clear set of deployment-relevant findings. Decoder LLMs produce substantially more stable explanations than encoder baselines, with 73% lower flip rates on average, and explanation stability improves further with model scale, showing a 44% gain from 7B to 70B parameters. The decoder advantage is most pronounced under disruptive perturbations such as deletion and reordering that commonly occur in real enterprise text streams, while semantic-preserving perturbations yield low flip rates for both model families. Finally, by relating stability gains to inference cost, our results form a practical cost–robustness tradeoff that supports model and explanation selection in compliance-sensitive enterprise deployments. Figure 1: Four-stage black-box evaluation pipeline. Six models across encoder and decoder families are accessed through prediction APIs; multi-level perturbations (character/word/sentence at three severity levels) are applied; a unified LOO occlusion explainer produces importance scores for original and perturbed inputs; Flip Rate and scaling analysis yield deployment guidance. Stages correspond to §2.1–§2.4. 2 Methodology Our methodology enables a fair comparison of explanation robustness across encoder and decoder architectures while reflecting realistic enterprise deployment constraints. We assume that models are typically served through governed endpoints or third-party APIs, where only input–output access is available and internal signals (e.g., gradients or hidden states) cannot be queried. Under this setting, we evaluate robustness by (i) generating user-like input perturbations, (ii) producing token-level explanations via a single black-box explainer that applies to both architectures, and (iii) measuring the stability of explanations between original and perturbed inputs with statistically grounded confidence intervals. 2.1 Black-box Evaluation Setting Let \mathcal{X} denote the space of natural-language texts and \mathcal{Y}=\{y_{1},\ldots,y_{K}\} the label set. A model M is accessed only through a prediction interface that returns a probability distribution over labels: M:\mathcal{X}\rightarrow\Delta(\mathcal{Y}), (1) where \Delta(\mathcal{Y}) is the probability simplex over \mathcal{Y} . We denote the predicted class by \hat{y}(x)=\arg\max_{y\in\mathcal{Y}}P(y\mid x), (2) and its confidence by P(\hat{y}(x)\mid x) . The black-box constraint prohibits access to model parameters \theta , intermediate representations h_{\ell} , gradients \nabla_{\theta} or \nabla_{x} , attention weights A , and any other internal attribution signals. This assumption matches enterprise model-serving environments where access is restricted for security, IP, or compliance reasons. It also ensures comparability across architectures: encoder models expose a classifier head with direct label logits, whereas decoder LLMs expose autoregressive token distributions; allowing white-box access would enable architecture-specific explainers and confound cross-family comparisons. To minimize variation unrelated to perturbations, we run inference deterministically whenever possible. For perturbations and bootstrapping (which rely on randomness), we fix global random seeds. For decoder LLMs, we use deterministic decoding (e.g., greedy decoding / temperature 0 ) so that changes in explanations are attributable to input modifications rather than sampling noise. 2.2 Perturbation Framework We define a perturbation operator \pi_{t,s}:\mathcal{X}\rightarrow\mathcal{X} parameterized by perturbation type t\in\mathcal{T} and severity s\in\mathcal{S} . Given an original input x , a perturbed input is \tilde{x}=\pi_{t,s}(x) . We set \mathcal{S}=\{0.05,0.10,0.20\} , controlling the fraction of positions modified for character- and word-level perturbations. We consider a taxonomy of perturbations spanning three linguistic levels. Character-level perturbations simulate typos and minor edit noise common in enterprise text streams such as tickets, notes, and free-form form inputs. The adjacent-character swap operator exchanges two neighboring characters at randomly selected positions, while character deletion removes randomly selected characters. For an input with C characters, each character-level operator modifies \lfloor s\cdot C\rfloor positions, subject to boundary and validity constraints. Word-level perturbations simulate small edits and partial rewrites. Synonym replacement substitutes selected tokens using WordNet 17 ; random word deletion removes selected tokens; and local word shuffling

Read the complete text