Latent Chain-of-Thought Improves Structured-Data Transformers
Carson Dudley, Samet Oymak · 2026
Read Latent Chain-of-Thought Improves Structured-Data Transformers by Carson Dudley, Samet Oymak (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-11262.txt.
Introduction
Carson Dudley and Samet Oymak show that latent chain-of-thought lifts structured-data transformers. Chain-of-thought and test-time compute aid reasoning. It explores latent CoT, depth, and looping for time-series and tabular data.
The team proposes a recurrent scheme: after a first pass, a transformer compresses query-position hidden states into feedback tokens. These tokens rejoin the input for multiple latent rounds before prediction. CoT models face a same-depth no-CoT baseline, a deeper baseline matched in depth, and a looped transformer with weight-tied recurrence and no extra CoT tokens. Across 36 datasets, latent CoT beats baseline on 7 of 9 time-series sets, gaining on average twelve point six three percent. On 23 of 27 tabular sets, it gains on average three point two five percent. Latent CoT lifts nanoTabPFN above the much larger TabPFN-v2 on TabArena. Modelers get reasoning gains without extra parameters.
Catalog record
| CREATOR | Carson Dudley, Samet Oymak |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 39778 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2605.11262
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Latent Chain-of-Thought Improves Structured-Data Transformers Carson Dudley, Samet Oymak Abstract Chain-of-thought and more broadly test-time compute are known to augment the expressive capabilities of language models and have led to major innovations in reasoning. Motivated by this success, this paper explores latent chain-of-thought as well as the impact of depth and looping for time-series and tabular data. We propose a recurrent scheme in which a structured-data transformer, after an initial forward pass, compresses its query-position hidden states into feedback tokens that are appended to the input and processed again, allowing multiple rounds of latent computation before prediction. We compare CoT models against a same-depth no-CoT baseline, a deeper baseline matched to the CoT model in effective depth, and a looped transformer with weight-tied recurrence but no additional chain-of-thought tokens. Across 36 datasets in time-series forecasting and tabular prediction, latent chain-of-thought improves over the baseline on 7/9 time-series datasets (+12.63% average gain) and 23/27 tabular datasets (+3.25% average gain), with CoT models performing best on average in both settings. We also show that the benefit of CoT extends to pretrained foundation models: applying latent CoT to nanoTabPFN, a small open-source tabular foundation model, improves its performance above the much larger TabPFN-v2 on TabArena. Together, these results demonstrate that chain-of-thought is a useful axis for scaling test-time compute for structured data. \cdots context query appended f_{\theta} H^{(r)}_{q} \phi_{\theta} Z^{(r)} append, repeat R times g_{\theta} \hat{y} prediction after R recurrences Figure 1 : Latent chain-of-thought for structured data. The transformer f_{\theta} runs on a sequence of context tokens, query tokens, and (after the first pass) appended feedback tokens. Query-position hidden states H^{(r)}_{q} are compressed by an MLP \phi_{\theta} into feedback tokens Z^{(r)} , which are appended to the sequence for the next pass. After R recurrences, the prediction head g_{\theta} maps from the hidden states to the prediction \hat{y} . 1 Introduction One of the main vehicles for recent progress in large language models has been scaling test-time compute. Reasoning models such as OpenAI’s o1 [ 1 ] , DeepSeek-R1 [ 2 ] , and their successors demonstrate that allocating more computation per query can substitute for, and sometimes outperform, scaling parameters [ 3 ] . One of the main mechanisms for scaling test-time compute is chain-of-thought reasoning: inducing the model to generate intermediate reasoning tokens in natural language to think through the problem [ 4 ] . It is not clear whether this reasoning has to happen in natural language. Most tokens in a chain of thought serve textual coherence rather than computation, and decoding to a discrete token at each step discards most of the information in the model’s hidden state. [ 5 ] address this directly with Coconut, which feeds the model’s last hidden state back as the next input embedding rather than decoding it to a single word, replacing discrete token-based chain-of-thought with a chain of latent thoughts. [ 6 ] achieve a similar goal through recurrent depth: rather than extending the sequence, the model iteratively applies a shared transformer block to refine its internal latent state before producing an output. This enables additional test-time computation without requiring chain-of-thought supervision or longer context. [ 7 ] further show that latent chain-of-thought enables parallel exploration of multiple reasoning paths, demonstrating additional flexibility of this paradigm for test-time scaling. Both of these approaches build on earlier work on weight-tied depth recurrence in the Universal Transformer [ 8 ] and looped transformers [ 9 ] , as well as papers showing that even content-free tokens can be used to allocate additional forward-pass computation [ 10 ] . All of this work has focused on language. Yet the same learning setup (a transformer doing in-context learning) is now used by state-of-the-art models for structured data. In tabular prediction, TabPFN [ 11 ] introduced in-context learning (ICL) over labeled rows, and its successors TabPFNv2 [ 12 ] and TabICL [ 13 ] use alternating row and column attention to scale this paradigm to medium and large tables, challenging the long-standing dominance of gradient-boosted trees [ 14 ] . In time-series forecasting, transformer-based foundation models such as Chronos [ 15 ] , TimesFM [ 16 ] , and Moirai [ 17 ] , as well as some domain-specific foundation models [ 18 , 19 ] , have become the standard. In both tabular prediction and time-series forecasting the model is a transformer that attends over a structured set of observations and produces predictions in a single forward pass—the same setting where adding latent computation has paid off in language modeling . Recent mechanistic analysis of tabular foundation models suggests that iterative computation may already be doing significant work inside these architectures. [ 20 ] probes state-of-the-art tabular foundation models (TabPFN, TabICL, LimiX) and find substantial depth-wise redundancy, with middle and late layers performing overlapping refinement of the residual stream rather than learning distinct transformations. They show that a single transformer block looped six times recovers the performance of a full six-layer model with only 20% of the parameters. This suggests that depth in tabular foundation models is primarily a vehicle for iterative refinement—which raises the question of whether explicit test-time iteration can extract further gains. In this paper, we ask whether latent chain of thought can transfer to structured data. Our approach gives a structured-data transformer multiple rounds of latent computation before it makes a prediction. After an initial forward pass, we take the hidden states at the prediction positions, compress them through a small MLP into feedback tokens, append those tokens to the model’s input representation, and run the transformer again. To enable controlled comparisons, we train each model from scratch on a single dataset rather than pretraining a separate foundation model for each recurrence variant. We compare against (i) a no-recurrence baseline at the same depth, (ii) a deeper baseline matched in total compute, isolating the contribution of latent recurrence from the confounds of pretraining scale and parameter count, and (iii) looped-transformer baselines that share weights across passes but lack the explicit feedback-token mechanism (i.e., recurrent passes with no CoT tokens) [ 9 ] . Training from scratch makes the comparison between mechanisms clean, but does not demonstrate whether latent CoT still helps once a foundation model has been pretrained at scale. To address this, we additionally pretrain nanoTabPFN [ 21 ] , a small open-source implementation of the TabPFN-v2 architecture, with latent CoT. On the TabArena small-table benchmark used in the nanoTabPFN paper [ 22 ] , our pretrained nanoTabPFN-CoT models surpass full TabPFN-v2 despite using only a 3-layer transformer with a small embedding dimension, suggesting that latent CoT can partially compensate for model scale even in the pretrained regime. We adapt latent chain of thought to tabular prediction and time-series forecasting, evaluate it on nine time-series datasets from LTSF-9 [ 23 ] and a 27-dataset subset of OpenML that was used in TabPFN evaluations [ 24 ] , and characterize when our approach improves prediction relative to both same-depth and matched-compute baselines. We additionally evaluate latent CoT in the pretrained foundation-model setting on TabArena [ 22 ] . Table 1 : Aggregate benchmark performance across structured-data tasks. “Best CoT” uses the CoT depth selected using validation performance from \{1,2,4\} independently for