EvasionBench: A Large-Scale Benchmark for Detecting Managerial Evasion in Earnings Call Q&A

Shijian MA, Yan LIN, Yi YANG · 2026

Read EvasionBench: A Large-Scale Benchmark for Detecting Managerial Evasion in Earnings Call Q&A by Shijian MA, Yan LIN, Yi YANG (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-09142.txt.

Introduction

Shijian Ma, Yan Lin, and Yi Yang build a test set to catch firms that dodge hard asks on earn calls. On such calls, chiefs take asks from pros on sales and risks. A chief may dodge: talk round the ask, give a half reply, or shift theme. Those dodges can hide weak spots that share holders need to know.

Their set is called EvasionBench. It draws on a vast pool of call ask and reply pairs from a pro data base. A strict sieve keeps clear cases. Each case is tagged by dodge grade: direct, mid, or full dodge. A reader in finance gets a scale to grade dodge find tools, and to flag calls for a close read.

Catalog record

CREATORShijian MA, Yan LIN, Yi YANG
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE25 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2601-09142

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

EvasionBench: A Large-Scale Benchmark for Detecting Managerial Evasion in Earnings Call Q&A Shijian MA, Yan LIN, Yi YANG Abstract We present EvasionBench , a comprehensive benchmark for detecting evasive responses in corporate earnings call question-and-answer sessions. Drawing from 22.7 million Q&A pairs extracted from S&P Capital IQ transcripts, we construct a rigorously filtered dataset and introduce a three-level evasion taxonomy: direct , intermediate , and fully evasive . Our annotation pipeline employs a Multi-Model Consensus (MMC) framework, combining dual frontier LLM annotation with a three-judge majority voting mechanism for ambiguous cases, achieving a Cohen’s Kappa of 0.835 on human inter-annotator agreement. We release: (1) a balanced 84K training set, (2) a 1K gold-standard evaluation set with expert human labels, and (3) Eva-4B , a 4-billion parameter classifier fine-tuned from Qwen3-4B that achieves 84.9% Macro-F1, outperforming Claude Opus 4.5, GPT-5.2, and Gemini 3 Flash. Our ablation studies demonstrate the effectiveness of multi-model consensus labeling over single-model annotation. EvasionBench fills a critical gap in financial NLP by providing the first large-scale benchmark specifically targeting managerial communication evasion. 1 Introduction Evasive communication—responding to questions without truly answering them—is a pervasive phenomenon across human discourse. Politicians deflect uncomfortable questions in interviews ( 7 ; 9 ) , witnesses hedge in legal depositions ( 2 ) , and corporate executives sidestep analyst inquiries during earnings calls ( 12 ; 10 ) . Detecting such evasion has concrete implications: in politics, it undermines democratic accountability; in legal contexts, it obscures truth-finding; and in financial markets, evasive management communication predicts subsequent earnings misses and stock underperformance ( 4 ; 18 ) . Despite evasion’s ubiquity, it remains underexplored in NLP. Unlike sentiment analysis—which has canonical benchmarks like SST and Financial PhraseBank—or question answering with SQuAD, no large-scale benchmark exists for evasion detection. This gap stems from two challenges: (1) the inherently subjective nature of “evasiveness,” which lies on a spectrum from direct to fully evasive, and (2) the prohibitive cost of expert annotation at scale. While sentiment captures what is said, evasion detection addresses whether the question was actually answered—a fundamentally different, discourse-level phenomenon rooted in Gricean pragmatics ( 11 ) . Corporate earnings calls offer an ideal testbed for evasion research. They provide: (a) high-stakes, adversarial Q&A with clearly defined speaker roles (analysts vs. executives), (b) abundant publicly available transcripts spanning decades, and (c) well-defined question types (quantitative, temporal, binary, causal) that enable operationalization of “directness.” Prior work has examined non-answers in this domain ( 10 ; 3 ) , but datasets remain limited in scale. In this paper, we introduce EvasionBench , the first large-scale benchmark for evasion detection, using earnings calls as our primary domain. While our data is domain-specific, we hypothesize that both our annotation framework and taxonomy could transfer to other adversarial Q&A settings (e.g., political interviews, legal depositions), though cross-domain validation remains future work. Our contributions are: • EvasionBench , a large-scale dataset comprising 84K balanced training samples and 1K human-validated evaluation samples, derived from 22.7M raw Q&A pairs spanning 2002–2022. • A Multi-Model Consensus (MMC) framework for scalable annotation that combines dual frontier LLM labeling with three-judge arbitration, achieving Cohen’s Kappa of 0.835 against human annotators. • Eva-4B , a 4-billion parameter classifier that achieves 84.9% Macro-F1, outperforming Claude Opus 4.5, GPT-5.2, and Gemini 3 Flash on this task. • Comprehensive analysis demonstrating that multi-model consensus labeling significantly outperforms single-model annotation, with ablations showing +4.3 pp Macro-F1 improvement. 2 Related Work 2.1 Evasion Across Domains Evasive communication has been studied extensively across multiple fields, though NLP benchmarks remain scarce. In political communication, 7 identified 43 distinct techniques politicians use to avoid answering questions, while 9 distinguished between overt evasion (explicit refusal) and covert evasion (appearing to answer while sidestepping the core). 5 theorized that equivocation arises from avoidance-avoidance conflicts where all direct answers carry costs—a framework directly applicable to corporate executives facing analyst scrutiny. In legal and forensic contexts, researchers have developed computational approaches to detect deceptive language in testimony ( 2 ; 30 ) . 22 identified linguistic markers of deception using LIWC, achieving 61–67% classification accuracy. Crucially, evasion differs from deception: while deception involves false statements, evasion represents strategic non-responsiveness where speakers appear cooperative but avoid the question core ( 24 ) . From a pragmatic perspective, evasion can be analyzed as a violation of Grice’s maxim of Relation ( 11 ) , often realized through hedging devices ( 16 ; 14 ) and indirect speech acts ( 27 ) . 6 connected such indirectness to face-threatening act mitigation, explaining why speakers strategically employ vague language. Despite this rich theoretical foundation across disciplines, NLP lacks a canonical benchmark for evasion detection—unlike sentiment ( 28 ) , stance ( 21 ) , or factual QA ( 26 ) . Our work addresses this gap by providing the first large-scale, systematically annotated evasion benchmark. 2.2 Evasion in Financial Communication Research on managerial evasion during earnings calls has grown substantially in accounting and finance. 12 provide empirical evidence that managers strategically choose what to disclose, with silence interpreted negatively by investors. 10 find that approximately 11% of analyst questions receive non-answers, with product-related questions frequently evaded in competitive environments. 3 offer theoretical frameworks for understanding why managers evade questions, while 4 examine how investor suspicion interacts with evasive communication. 23 propose a psychological taxonomy for evasive answers; however, their dataset relies solely on single-model LLM annotation without human validation, raising concerns about label reliability and potential model-specific biases. 2.3 Financial NLP Financial text analysis has evolved from lexicon-based methods ( 20 ) to neural approaches. 1 introduced FinBERT for financial sentiment analysis, demonstrating the value of domain-specific pretraining. 15 predict earnings surprises from conference call transcripts using LLMs. 13 show that managers strategically manage linguistic tone. However, these works focus on sentiment or prediction rather than discourse-level phenomena like evasion, which requires assessing whether a response actually addresses the question asked. 2.4 Scalable Annotation with LLMs The prohibitive cost of expert annotation has driven interest in LLM-based labeling approaches. Using LLMs as evaluators has gained traction for scalable assessment ( 32 ) , though 8 identify systematic biases including position bias and verbosity preference. Several methodologies address annotation quality and consistency. Self-consistency ( 31 ) samples multiple reasoning paths from a single model. RLAIF ( 19 ) uses AI feedback to scale preference learning. Tri-training ( 33 ) leverages three classifiers to label unlabeled data through agreement. Recent surveys on LLM-based annotation ( 29 ) validate consensus-based approaches for improving label quality. Our MMC framework draws on these insights, combining multiple heterogeneous frontier LLMs (rather than homogeneous classifiers) with randomized presentation order to mitigate individual model biases. This design choice is motivated by our finding that different LLMs exhibit distinct labeling tendencies (Section 5 ). 2.5 Summary and Positioning Table 1 synthesizes prior work on evasion detection in financial communication. Three gaps emerge: (1) existing datasets lack scale—the largest prior corpus contains fewer than 15K samples with limited human validation; (2) annotation often relies on single-model LLM labeling without consensus mechanisms, introducing systematic bias; (3) no prior work provides both large-scale training data and rigorously human-validated evaluation sets. EvasionBench addresses all three limitations. Work Scale Labels Human Multi-M 10 2.1K 2

Read the complete text