ReSS: Learning Reasoning Models for Tabular Data Prediction via Symbolic Scaffold

Chenlang Yi, Gang Li, Zizhan Xiong, Tue Minh Cao, Yanmin Gong, My T. Thai, Tianbao Yang · 2026

Read ReSS: Learning Reasoning Models for Tabular Data Prediction via Symbolic Scaffold by Chenlang Yi, Gang Li, Zizhan Xiong, Tue Minh Cao, Yanmin Gong, My T. Thai, Tianbao Yang (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-13392.txt.

Introduction

Yi and six colleagues build ReSS, a system that teaches language models to reason faithfully over tables in high-stakes fields like health care and finance. Symbolic models give checkable logic but lack rich meaning, while general models need special tuning for tables. ReSS uses a decision tree to pull out decision paths as symbolic scaffolds, then guides a language model to write reasoning that sticks to that logic. Fine-tuning on the resulting data, plus scaffold-invariant augmentation, lifts results over trees and standard tuning by up to 10 percent.

Catalog record

CREATORChenlang Yi, Gang Li, Zizhan Xiong, Tue Minh Cao, Yanmin Gong, My T. Thai, Tianbao Yang
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE112182
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2604.13392

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

ReSS: Learning Reasoning Models for Tabular Data Prediction via Symbolic Scaffold Chenlang Yi, Gang Li, Zizhan Xiong, Tue Minh Cao, Yanmin Gong, My T. Thai, Tianbao Yang Abstract Tabular data remains prevalent in high-stakes domains such as healthcare and finance, where predictive models are expected to provide both high accuracy and faithful, human-understandable reasoning. While symbolic models offer verifiable logic, they lack semantic expressiveness. Meanwhile, general-purpose LLMs often require specialized fine-tuning to master domain-specific tabular reasoning. To address the dual challenges of scalable data curation and reasoning consistency, we propose ReSS , a systematic framework that bridges symbolic and neural reasoning models. ReSS leverages a decision-tree model to extract instance-level decision paths as symbolic scaffolds. These scaffolds, alongside input features and labels, guide an LLM to generate grounded natural-language reasoning that strictly adheres to the underlying decision logic. The resulting high-quality dataset is used to fine-tune a pretrained LLM into a specialized tabular reasoning model, further enhanced by a scaffold-invariant data augmentation strategy to improve generalization and explainability. To rigorously assess faithfulness, we introduce quantitative metrics including hallucination rate, explanation necessity, and explanation sufficiency. Experimental results on medical and financial benchmarks demonstrate that ReSS-trained models improve traditional decision trees and standard fine-tuning approaches up to 10 % 10\% while producing faithful and consistent reasoning. 1 Introduction In high-stakes domains such as healthcare ( 12 ) , finance ( 24 ) where tabular data is common, predictive accuracy alone is often insufficient. Models are also expected to provide faithful and human-understandable reasoning that enables practitioners to inspect, trust, and act upon their predictions. Despite decades of progress in tabular learning ( 7 ; 10 ; 26 ) , constructing models that jointly achieve strong performance and meaningful reasoning remains a fundamental challenge. Traditional machine learning methods have long dominated tabular prediction, with tree-based approaches such as decision tree ( 7 ) , random forests ( 8 ) , XGBoost ( 10 ) , and LightGBM ( 20 ) achieving strong empirical performance. From a reasoning perspective, however, these models remain fundamentally limited. Tree ensemble methods do not expose an explicit, instance-level decision process, while single decision trees express reasoning solely through symbolic rules defined over low-level feature thresholds. Although such representations faithfully reflect the model’s internal logic, they lack semantic grounding in domain concepts and therefore cannot be readily interpreted by non-expert users, limiting their practical utility in real-world, high-stakes decision-making settings. Recent advances in deep learning (DL) for tabular data have produced models that are able to match or even surpass the performance of tree-based methods ( 18 ; 3 ; 1 ; 26 ) . However, these models usually provide limited or no interpretability regarding its decision process. Even when post-hoc explanations are available, they often fail to constitute truly human-understandable reasoning chains. Recently, studies have leveraged LLMs for tabular data prediction ( 13 ; 15 ) . These methods transform tabular rows into natural language prompts to utilize the LLM’s prior knowledge for prediction. However, they essentially function as direct-prediction models rather than reasoning models, as they map inputs directly to final labels while bypassing the intermediate decision-making process. Consequently, they fail to provide a transparent, step-by-step logical chain to justify their predictions. This raises an important question: How to efficiently train a faithful reasoning model for tabular data prediction? There are two key challenges to be addressed: (i) how to curate high-quality reasoning data for fine-tuning a LLM; and (ii) how to ensure that the reasoning traces generated by the learned model are both faithful and explainable. Curating high-quality reasoning data for high-stakes domains such as healthcare and finance typically requires domain experts and is therefore expensive, time-consuming, and difficult to scale. Moreover, while current LLMs are capable of producing chain-of-thought (CoT)-like outputs, these explanations are not guaranteed to be faithful and causal to the tabular data prediction ( 19 ; 5 ) . To address these challenges, we propose ReSS , a systematic framework that curates high-quality reasoning data via symbolic scaffolds and trains efficient reasoning models for tabular prediction. ReSS first trains a decision tree on tabular data and extracts instance-level decision paths from samples that are correctly predicted by the decision tree, which serve as explicit symbolic scaffolds capturing the model’s underlying decision logic. These scaffolds, together with input features and labels, are used to guide an LLM to generate grounded, step-by-step natural-language reasoning that strictly adheres to the logical constraints encoded in the decision paths, yielding a reasoning dataset that is both faithful to the original decision process and interpretable by humans. To further enhance generalization and explainability, ReSS incorporates a scaffold-invariant data augmentation strategy that expands the reasoning dataset and introduces out-of-distribution examples by perturbing input features while preserving the symbolic decision paths and labels. An overview of the ReSS pipeline is shown in Figure 1 . We summarize our contributions as follows: • We propose ReSS, a systematic framework that leverages decision-tree paths as symbolic scaffolds to guide an LLM using its inherent domain knowledge to curate high-quality grounded reasoning data, and trains efficient reasoning models for tabular prediction. • We introduce a scaffold-invariant data augmentation strategy that preserves symbolic decision logic while generating both in-distribution and out-of-distribution reasoning data, enabling improved generalization and more robust explainability. • We propose a set of metrics to measure the faithfulness and explainability of generated CoT-like outputs by the trained reasoning model, including hallucination rate, explanation sufficiency, and explanation necessity. • We conduct experiments on four real-world datasets, including two from medicine and two from finance. The results demonstrate the effectiveness of ReSS and the scaffold-invariant data augmentation, and validate the faithfulness and causal impact of the generated rationales. 2 Related work While tabular data prediction has long been dominated by tree-based ensemble models, recent years have seen the emergence of new approaches based on DL and LLMs. Deep Learning Methods for Tabular Prediction. Early DL approaches for tabular prediction typically rely on generic architectures such as MLPs, which lack inductive biases aligned with the structure of tabular decision manifolds and often struggle to match the performance of tree-based methods ( 3 ) . To overcome this limitation, subsequent work introduced tabular-specific inductive biases into neural architectures. TabNet ( 3 ) injects inductive bias through sequential, instance-wise feature selection, enabling competitive performance with tree-boosting models on a wide range of benchmarks. DNF-Net ( 1 ) incorporates a logical inductive bias by aligning model structure with disjunctive normal form, encouraging localized and rule-like decision behaviors. TabTransformer ( 18 ) leverages self-attention to model complex feature interactions, demonstrating that transformer-based architectures can match or surpass gradient-boosted trees in tabular prediction tasks. TabPFN ( 16 ) formulates tabular prediction as in-context inference with a prior-data fitted transformer network and achieves competitive or superior performance on small to medium-sized datasets. InterpreTabNet ( 26 ) attempts to provide intrinsic interpretability through instance-wise feature selection mechanisms, by employing an LLM to map from the learned feature mask onto natural language text describing the learned signal. Despite these advances, they do not aim to produce explicit, human-understandable reasoning processes. LLMs for Tabular Prediction. Recent work has explored applying LLMs to tabular prediction by serializing tabular records into natural-language descriptions, allowing LLMs

Read the complete text