Fortune : Formula-Driven Reinforcement Learning for Symbolic Table Reasoning in Language Models

Lang Cao Jingxian Xu Hanbing Liu Jinyu Wang Mengyu Zhou Haoyu Dong Shi Han Dongmei Zhang · 2025

Read Fortune : Formula-Driven Reinforcement Learning for Symbolic Table Reasoning in Language Models by Lang Cao Jingxian Xu Hanbing Liu Jinyu Wang Mengyu Zhou Haoyu Dong Shi Han Dongmei Zhang (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-23667.txt.

Introduction

Lang Cao and co-authors train language models to answer table questions by writing spreadsheet formulas, in a research paper from 2025. Models reason well in words but slip on numbers in complex tables. Formulas offer an exact, checkable path from data to answer.

Their method, called Fortune, rewards the model when its formula runs and lands the right answer. No hand-labeled formulas are needed for training. Across seven table benchmarks, the approach lifted performance sharply, above all on multi-step numeric tasks. The trained model, Formula-R1, shows that executable answers beat mental arithmetic for table work. The framework opens a practical route to reliable data assistants.

Catalog record

CREATORLang Cao Jingxian Xu Hanbing Liu Jinyu Wang Mengyu Zhou Haoyu Dong Shi Han Dongmei Zhang
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE87 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2505-23667

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Fortune : Formula-Driven Reinforcement Learning for Symbolic Table Reasoning in Language Models Lang Cao Jingxian Xu Hanbing Liu Jinyu Wang Mengyu Zhou Haoyu Dong Shi Han Dongmei Zhang Abstract Tables are a fundamental structure for organizing and analyzing data, making effective table understanding a critical capability for intelligent systems. While large language models (LMs) demonstrate strong general reasoning abilities, they continue to struggle with accurate numerical or symbolic reasoning over tabular data, especially in complex scenarios. Spreadsheet formulas provide a powerful and expressive medium for representing executable symbolic operations, encoding rich reasoning patterns that remain largely underutilized. In this paper, we propose Formula Tuning (Fortune), a reinforcement learning (RL) framework that trains LMs to generate executable spreadsheet formulas for question answering over general tabular data. Formula Tuning reduces the reliance on supervised formula annotations by using binary answer correctness as a reward signal, guiding the model to learn formula derivation through reasoning. We provide a theoretical analysis of its advantages and demonstrate its effectiveness through extensive experiments on seven table reasoning benchmarks. Formula Tuning substantially enhances LM performance, particularly on multi-step numerical and symbolic reasoning tasks, enabling a 7B model to outperform OpenAI o1 on table understanding. This highlights the potential of formula-driven RL to advance symbolic table reasoning in LMs. † † † Corresponding authors. Figure 1 : Overview of Formula Tuning ( Fortune ). 1 Introduction Tables are a common and practical data structure in daily life, playing a central role in data collection, representation, and analysis [ 1 , 2 ] . Recent advances in large language models (LLMs) [ 3 , 4 , 5 ] have brought impressive performance across a wide range of natural language processing tasks, including language understanding [ 6 , 7 ] and general reasoning [ 8 ] . Naturally, LLMs have also been applied to tabular data understanding and reasoning [ 9 , 10 , 11 ] . However, reasoning over tabular data remains a key challenge for language models (LMs) [ 11 ] , primarily because of the numerical nature and intricate structure of tables, which pose significant obstacles to understanding both content and layout. Besides, some findings [ 12 ] suggest current LLMs rely on pattern memorization over genuine rule learning, often leading to incorrect mathematical computations during traditional chain-of-thought reasoning, also referred to as textual reasoning. Some approaches introduce symbolic methods [ 13 , 14 ] , where models first generate symbolic representations (such as programs) and then execute them to obtain results. While this improves arithmetic accuracy, such methods often struggle to generalize due to limited symbolic reasoning or program generation capabilities. Instead of truly understanding the context and generating problem-solving programs, LMs often fall back on memorized code snippets from pretraining [ 15 ] . Since not all complex symbolic patterns can be memorized, and given that high-quality code supervision is scarce [ 16 ] , more effective strategies for symbolic table reasoning are needed. Recent progress in reinforcement learning (RL) for LMs shows promise for overcoming such limitations. For example, DeepSeek-R1 [ 17 ] improves mathematical reasoning in LLMs via rule-based RL rewards, without relying on step-by-step annotations. DeepRetrieval [ 18 ] uses retrieval metrics as RL rewards to train models to reason over queries that maximize real-world retrieval performance across search engines and databases. DeepCoder [ 19 ] also demonstrates the effectiveness of RL for code reasoning and generation. These works collectively demonstrate the promise of RL in enabling LMs to perform robust symbolic reasoning without explicit intermediate supervision. When it comes to symbolic table reasoning, spreadsheet formula [ 20 ] is a powerful and versatile tool. In real-world scenarios, tabular data is often stored in spreadsheet formats (e.g., Microsoft Excel, Google Sheets) [ 21 ] , where each cell holds individual data values. The spreadsheet formulas embedded in these files act as lightweight, program-like constructs that enable users to compute, transform, and reason over data. Compared to structured interfaces such as SQL or Python/Pandas, which are typically restricted to relational or flat table schemas, spreadsheet formulas offer greater flexibility, as they can be applied to arbitrary two-dimensional tables without structural constraints [ 22 ] . Moreover, spreadsheet formulas are Turing complete [ 23 ] , making them particularly well-suited as a medium for symbolic table reasoning. We envision that by training LMs to understand and generate spreadsheet formulas, they can acquire more robust and generalizable symbolic reasoning capabilities over tabular data. However, current LLMs still struggle to produce accurate and reliable spreadsheet formulas, as highlighted by recent evaluations [ 24 ] . At the same time, existing publicly available spreadsheet datasets tend to include relatively simple formulas [ 25 ] , rely on heuristic conversions from SQL [ 26 ] , or synthesize formulas using LLMs in constrained question-answering settings [ 22 ] . These approaches fall short of capturing the complexity necessary for diverse symbolic reasoning tasks and real-world downstream applications. This limitation poses a significant barrier to effectively leveraging spreadsheet formulas for training LMs in symbolic table reasoning. To address these challenges, we propose For mula Tun ing ( Fortune ), a RL framework designed to teach LMs to perform symbolic reasoning over general tabular data through spreadsheet formula. Specifically, our framework leverages answer correctness of formula execution results as a reward signal to guide the LMs in deriving formulas through reasoning (Figure 1 ). This approach reduces reliance on supervised formula annotations and enables LMs to generate executable formulas that answer questions over tables with improved accuracy. Extensive experiments validate the effectiveness of Formula Tuning , demonstrating that RL is more effective than supervised fine-tuning (SFT) in enhancing the symbolic reasoning capabilities of LMs. We also find that initializing RL with SFT as a cold start [ 17 ] provides a stronger foundation and raises the upper bound of RL performance, with SFT serving as a form of knowledge injection. Notably, this enables a 7B model to outperform OpenAI o1 in overall performance (68.48% vs. 66.90%) . Furthermore, we train both textual and symbolic reasoning components using RL in Fortune++ . By jointly leveraging both components during inference, our method achieves strong performance across multiple benchmarks (e.g., 82.54% on WikiTQ, 95.06% on TabFact, 87.24% on HiTab, 80.47% on FinQA, and 93.20% on AIT-QA) . In summary, this paper makes the following key contributions: • We propose Formula Tuning ( Fortune ), a reinforcement learning framework that enhances symbolic reasoning for table understanding by training language models to generate executable spreadsheet formulas. • We provide a theoretical analysis and discussion comparing textual versus symbolic reasoning in table understanding, as well as supervised fine-tuning versus reinforcement learning in symbolic table reasoning. • We conduct extensive experiments on seven table understanding benchmarks, demonstrating the effectiveness of Formula Tuning , and perform comprehensive analyses to provide deeper insights. 2 Related Work Table Understanding and Reasoning. Many studies have explored fine-tuning language models (LMs) to improve their ability to understand and reason over tabular data. Building on the masked language modeling introduced by BERT [ 27 ] , models such as TaPas [ 28 ] , PaSTA [ 29 ] , and TUTA [ 30 ] propose specialized pretraining strategies tailored for tables. TAPEX [

Read the complete text