Evaluating LLM Understanding via Structured Tabular Decision Simulations
Sichao Li [email protected], Xinyue Xu [email protected], Xiaomeng Li [email protected] · 2025
Read Evaluating LLM Understanding via Structured Tabular Decision Simulations by Sichao Li [email protected], Xinyue Xu [email protected], Xiaomeng Li [email protected] (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2511-10667.txt.
Introduction
Sichao Li, Xinyue Xu, and Xiaomeng Li argue that getting answers right is not the same as understanding, for language models any more than for people. They introduce Structured Tabular Decision Simulations, a suite of expert-style decision exams that test whether a model bases its choices on the factors that actually determine outcomes in a domain.
The suite jointly checks question comprehension, knowledge-based prediction, and reliance on relevant decision factors. Running nine frontier models across fifteen decision settings, they find most models cannot sustain strong accuracy across domains. Worse, models can be accurate yet globally unfaithful, with stated rationales frequently mismatching the factors driving their predictions. The authors call for evaluation protocols that measure understanding at a global level, beyond accuracy alone.
Catalog record
| CREATOR | Sichao Li [email protected], Xinyue Xu [email protected], Xiaomeng Li [email protected] |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 101 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2511-10667
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Evaluating LLM Understanding via Structured Tabular Decision Simulations Sichao Li [email protected], Xinyue Xu [email protected], Xiaomeng Li [email protected] Abstract Large language models (LLMs) often achieve impressive predictive accuracy, yet correctness alone does not imply genuine understanding . True LLM understanding, analogous to human expertise, requires making consistent, well-founded decisions across multiple instances and diverse domains, relying on relevant and domain-grounded decision factors. We introduce Structured Tabular Decision Simulations (STaDS) , a suite of expert-like decision settings that evaluate LLMs as if they were professionals undertaking structured decision “exams”. In this context, understanding is defined as the ability to identify and rely on the correct decision factors , features that determine outcomes within a domain. STaDS jointly assesses understanding through: (i) question and instruction comprehension, (ii) knowledge-based prediction, and (iii) reliance on relevant decision factors. By analyzing 9 frontier LLMs across 15 diverse decision settings, we find that (a) most models struggle to achieve consistently strong accuracy across diverse domains; (b) models can be accurate yet globally unfaithful , and there are frequent mismatches between stated rationales and factors driving predictions. Our findings highlight the need for global-level understanding evaluation protocols and advocate for novel frameworks that go beyond accuracy to enhance LLMs’ understanding ability. 1 Introduction Large language models (LLMs) are increasingly deployed as surrogate professionals due to their strong predictive performance, acting as physicians for medical triage, analysts for financial risk assessment, or policy advisors for legislative decisions 1 ; 13 ; 24 ; 69 . In such applications, users expect models to reason and make decisions with the reliability of domain experts. Yet current evaluations overwhelmingly focus on surface metrics like accuracy or task completion. What is largely missing is an assessment of the model’s understanding ability : its internal competence to grasp and apply the principles that govern a decision task, going beyond making a single correct prediction . Understanding in this context is cognitive rather than purely behavioral, which makes it challenging to measure explicitly. While recent work has begun to ask whether LLMs “reason faithfully”, especially through chain-of-thought (CoT) rationales, such analyses mostly operate at the level of a single problem instance: does the explanation accompanying an answer reflect the steps that actually produced it? ( 60 ; 38 ; 9 ; 66 ) These studies expose important failures of local reasoning faithfulness, but they do not tell us whether a model behaves like a reliable expert across many decisions in a domain ( 32 ; 7 ) . In this work, we extend the concept of faithfulness from local reasoning traces to global decision faithfulness, exploring whether the model’s predictions across multiple cases consistently rely on a governing rule, driven by meaningful decision factors, rather than superficial correlations (see our motivation in Fig. 1 ). Figure 1 : This diagram illustrates how the STaDS protocol simulates expert decision-making processes in structured tabular decision settings. The protocol evaluates LLMs’ understanding ability through three key dimensions: (1) Question and Instruction Comprehension, assessing task interpretation and output adherence; (2) Knowledge-based Prediction, evaluating the model’s application of domain knowledge for accurate predictions; and (3) Reliance on the Right Decision Factors, determining whether predictions align with the factors the model claims to rely on. The diagram depicts how these dimensions together form a principled basis for understanding and evaluating LLMs. This limitation motivates a shift in perspective. We propose evaluating the understanding ability of LLMs as a distinct axis of competence by simulating human-like decision settings, through the lens of interpretability and explainability. By understanding ability, we refer to the capacity of an LLM to capture the underlying concepts of a task, generalize across diverse instances, and base its decisions on conceptually meaningful features ( 43 ; 11 ) . Human experts exemplify this ability: they demonstrate understanding not merely by producing a correct answer once, but by consistently applying underlying domain concepts across diverse cases. For instance, a physician is expected to base diagnoses on established medical knowledge rather than incidental correlations. Similarly, a reliable LLM should leverage its pre-learned knowledge representations to generate well-founded, conceptually grounded decisions. To probe this dimension, we introduce Structured Tabular Decision Simulations (STaDS) protocol: a systematic evaluation framework that casts expert-like decision problems into tabular form (details see Sec. 3.2 ), enabling controlled assessment of both predictive performance and explanation faithfulness. Unlike reasoning, which emphasizes step-by-step justifications where models may make errors on intermediate steps ( 55 ) , tabular simulations focus on an end-to-end evaluation on whether LLMs capture the underlying decision rules or policies that govern outcomes ( interpretability ). Moreover, tabular features and labels are unambiguous and semantically well-defined: each corresponds to a clearly grounded concept (e.g., disease diagnosis, loan default, voting preference) that has direct meaning in the application domain. This stands in contrast to free-text rationales , which may admit multiple plausible interpretations, or image-based tasks , where concepts are often loosely bounded and harder to specify, and typically require extensive labeling, human validation, and additional grounding efforts to ensure explainability ( 35 ; 67 ; 40 ) . In this way, the setting enables quantitative assessment of understanding ability. Research Questions and Contributions. Ultimately, we target two central research questions: RQ1: To what extent do LLMs demonstrate understanding ability by generalizing across diverse tabular decision settings, beyond surface-level correlations? RQ2: To what extent do LLMs demonstrate globally faithful behavior, consistently identifying the decision factors that govern outcomes within a domain? To answer these questions, we present a unified evaluation framework and empirical study of LLMs’ understanding ability. Our primary contributions are as follows: • STaDS Protocol: We propose the Structured Tabular Decision Simulation (STaDS) protocol as a comprehensive evaluation of LLMs’ understanding ability through structured, expert-like decision settings. STaDS operationalizes understanding along three complementary axes: (i) Question and instruction comprehension : the ability to interpret task instructions and follow structured output specifications; (ii) Knowledge-based prediction : the capacity to apply intrinsic and in-context knowledge to produce accurate predictions; and (iii) Reliance on the right decision factors : the degree to which a model’s predictions are driven by the same decision factors it claims to rely on. Unlike conventional evaluation frameworks, STaDS enables an end-to-end assessment that minimizes prompt bias and avoids errors from unfaithful reasoning steps. It provides a structured, reproducible, and extensible setting for LLMs, bridging concepts of interpretability (internal mechanisms) and explainability (post-hoc justifications) from eXplainable AI (XAI) to LLM evaluation ( 10 ) . • STaDS Metrics and Benchmarks. Building on these three axes, we define targeted metric suites for each: (1) Comprehension Fidelity : captured by Len-F1, UnkLbl%, and the format-related component of Penalized Accuracy to quantify instruction adherence; (2) Predictive Competence : measured through zero/few-shot Accuracy, Macro-F1, and overall Penalized Accuracy to assess knowledge grounding; and (3) Decision Faithfulness : evaluated via LAO-based feature reliance, Self-Decision Faithfulness, and SelfAtt@k to determine whether stated and actual decision factors align at the global level. To support systematic evaluation, we release a curated suite of 15 real-world tabular datasets spanning healthcare, finance, and public policy, summing up to 160k tasks, where each task represents a real-world decision environment. This is accompanied by standardized instruction templates designed to minimize prompt bias and promote reproducibility. • Empirical