A Closer Look into LLMs for Table Understanding
Jia Wang, Chuanyu Qin, Mingyu Zheng, Qingyi Si, Peize Li, Zheng Lin · 2026
Read A Closer Look into LLMs for Table Understanding by Jia Wang, Chuanyu Qin, Mingyu Zheng, Qingyi Si, Peize Li, Zheng Lin (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-15402.txt.
Introduction
Wang and co-authors probe how large language models understand tables, testing sixteen models across general, tabular, and mixture-of-experts types. They track attention flow, layer depth, expert use, and input design. Attention moves in three phases: early layers scan wide, middle layers find key cells, late layers boost them. Table tasks need deeper layers than math tasks before predictions settle. Mixture models route to table experts in the middle layers. Chain-of-thought prompts raise table attention, and table tuning raises it further.
Catalog record
| CREATOR | Jia Wang, Chuanyu Qin, Mingyu Zheng, Qingyi Si, Peize Li, Zheng Lin |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 51170 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2603.15402
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
A Closer Look into LLMs for Table Understanding Jia Wang, Chuanyu Qin, Mingyu Zheng, Qingyi Si, Peize Li, Zheng Lin Abstract Despite the success of Large Language Models (LLMs) in table understanding, their internal mechanisms remain unclear. In this paper, we conduct an empirical study on 16 LLMs, covering general LLMs, specialist tabular LLMs, and Mixture-of-Experts (MoE) models, to explore how LLMs understand tabular data and perform downstream tasks. Our analysis focus on 4 dimensions including the attention dynamics, the effective layer depth, the expert activation, and the impacts of input designs. Key findings include: (1) LLMs follow a three-phase attention pattern—early layers scan the table broadly, middle layers localize relevant cells, and late layers amplify their contributions; (2) tabular tasks require deeper layers than math reasoning to reach stable predictions; (3) MoE models activate table-specific experts in middle layers, with early and late layers sharing general-purpose experts; (4) Chain-of-Thought prompting increases table attention, further enhanced by table-tuning. We hope these findings and insights can facilitate interpretability and future research on table-related tasks. Our code and analysis pipeline is publicly available at https://github.com/JiaWang2001/closer-look-table-llm † † ∗ Indicates equal contribution. † † ‡ Corresponding author: Zheng Lin. 1 Introduction Tables, as a representative form of structured data, are widely used across various real-world fields to store and present information. In recent years, Large Language Models (LLMs) have shown powerful instruction following and complex reasoning ability, and thus have become the dominant paradigm for table understanding technique, supporting a wide range of downstream application scenarios, such as table question answering (TQA) 32 ; 54 ; 10 , table fact verification (TFV) 5 ; 44 , advanced data analysis 24 ; 45 ; 31 and spreadsheet manipulation 23 ; 28 . Although remarkable success has been achieved with existing LLM-based table understanding approaches, they predominantly focus on how to continually improve performance metrics across downstream tasks, e.g., boosting LLMs’ performance with the most suitable prompt designs 41 ; 53 and developing specialist tabular LLMs with supervised fine-tuning (SFT) 10 ; 54 ; 55 or reinforcement learning (RL) 52 ; 47 ; 19 . Unlike previous performance-oriented work, in this paper, we conduct an in-depth empirical study to achieve a deeper understanding of the underlying mechanisms of LLMs’ table understanding. Our experiments cover a wide spectrum of LLMs of different sizes and types, and focuses on four perspectives including the attention dynamics, the effective layer depth, the impact of MoE architectures and the influence of input designs. The explored research questions are shown in below. Q1: How do intra-table and extra-table attention patterns of LLMs evolve when performing table understanding tasks? Q2: How many effective layers are utilized by LLMs to achieve stable predictions and are there difference between tabular and general tasks? Q3: Are there specialized table experts for MoE models and where are they located? Q4: The influence of different table formats and reasoning strategies on internal representations. We conclude our key findings as follows: (1) Different LLM layers exhibit distinct attention patterns during table understanding. The early layers broadly scan the overall table content, the middle layers then concentrate attention on query-related cells and the top layers further amplify the focused content representations to produce the final answer. (2) Tabular tasks require more layers for output distribution refinement than math reasoning. While the depth at which final answer content crystallizes is broadly comparable across tasks, LLMs refine output distributions across more layers when processing tabular data—a pattern consistent across model scales and training strategies. (3) MoE models develop specialized experts for tabular tasks in the middle layers, which show minimal overlap with math-oriented experts, but early and top layers share general-purpose experts across different tasks. (4) Input formats and reasoning strategies indeed influence attention patterns. HTML tables result in more dispersed attention distributions in early layers than Markdown tables. Chain-of-Thought (CoT) prompting can allocate more attention to the table content, which can be further amplified by table-specific fine-tuning and thus leads to better performance. To the best of our knowledge, we present the first thorough investigation of the mechanisms underlying LLM-based table understanding. Our findings not only provide new insights into LLM interpretability in the tabular domain, but also offer guidelines for future research—from optimal input configurations that maximize attention to relevant table content, to inference-time interventions in later layers, to MoE optimizations for table-specialized experts. 2 Related Work 2.1 LLM-based Table Understanding Table understanding (TU) technique aims to enable models to automatically comprehend tables and perform various tabular tasks based on user requests 35 ; 26 . With the rapid iteration of LLMs, their ability has opened new possibilities for more intelligent TU applications. One line of research endeavours to enhance LLMs’ table understanding ability through different strategies. For instance, eliciting correct reasoning steps with prompt engineering and carefully selected in-context examples 6 ; 53 ; 44 ; 16 ; 17 , collecting table instruction tuning data for supervised fine-tuning 54 ; 55 , exploring new tasks and reward strategies for reinforcement learning 52 ; 47 ; 57 , and building powerful table agents for multi-turn data analysis and excel manipulation 23 ; 45 . In addition to the performance-oriented studies mentioned above, another valuable direction is to explore the robustness and interpretability of LLMs’ table-related capabilities. For example, evaluating models’ performance against divergent perturbations such as table formats 41 , noisy operators 37 and cell replacement 4 . Compared with existing work that mainly focused on the performance robustness, we aim at seeking a better understanding of the underlying mechanisms of LLM-based table understanding and provide valuable insights for future follow-ups. 2.2 Interpretability and Analysis of LLMs Despite the strong capabilities of LLMs, understanding their internal mechanisms remains a key challenge. Some studies focus on the roles of different layers in LLMs, examining how individual layers contribute to information processing 38 and how semantic representations are gradually constructed across layers 8 ; 15 . Other lines of work investigate specific internal components, such as attention heads in self-attention mechanisms and feed-forward network (FFN) modules, with the aim of clarifying their roles in knowledge representation 13 ; 27 and information acquisition 46 ; 22 , revealing various intriguing phenomena within LLMs, such as attention sink 21 ; 49 ; 14 ; 33 ; 34 . Furthermore, with the growing adoption of Mixture-of-Experts (MoE) architectures, recent researches 43 ; 1 ; 25 ; 2 ; 40 analyze expert modules within MoE models, exploring their specialization patterns and activation behaviors during task execution. While these efforts have provided valuable insights 36 ; 3 , they predominantly focus on unstructured text tasks such as math reasoning and factual recall. In this work, we extend interpretability research to structured table data, conducting analyses to explore how LLMs process tables and offering preliminary findings across several dimensions. 3 Experimental Setups Evaluation Data We randomly select 500 samples from 3 TQA benchmarks and 1 TFV benchmark to perform empirical study, which include WTQ ( 32 ) , HiTab ( 7 ) , AIT-QA ( 20 ) and TabFact ( 5 ) . The resulting 2,000 test samples cover common table structures with flat headers (WTQ, TabFact) and complex hierarchical headers (HiTab, AIT-QA). The input tables are