Robustness is Important: Limitations of LLMs for Data Fitting

Hejia Liu, Mochen Yang, Gediminas Adomavicius · 2025

Read Robustness is Important: Limitations of LLMs for Data Fitting by Hejia Liu, Mochen Yang, Gediminas Adomavicius (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2508-19563.txt.

Introduction

Hejia Liu, Mochen Yang, and Gediminas Adomavicius warn in a 2025 paper that large language models are not robust tools for data fitting. Such models are now used well past language work, as a plug-and-play way to fit data and make predictions, and past tests show they can match classic methods on score.

The team found a sharp flaw. Changes to how data are shown, with no bearing on the task, can swing the model's predictions a lot. Just renaming the variables moved the size of the error by a large share in some tests. The paper frames robustness as a core need for such use.

Catalog record

CREATORHejia Liu, Mochen Yang, Gediminas Adomavicius
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE46 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2508-19563

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Robustness is Important: Limitations of LLMs for Data Fitting Hejia Liu, Mochen Yang, Gediminas Adomavicius Abstract Large Language Models (LLMs) are being applied in a wide array of settings, well beyond the typical language-oriented use cases. In particular, LLMs are increasingly used as a plug-and-play method for fitting data and generating predictions. Prior work has shown that LLMs, via in-context learning or supervised fine-tuning, can perform competitively with many tabular supervised learning techniques in terms of predictive performance. However, we identify a critical vulnerability of using LLMs for data fitting – making changes to data representation that are completely irrelevant to the underlying learning task can drastically alter LLMs’ predictions on the same data. For example, simply changing variable names can sway the size of prediction error by as much as 82% in certain settings. Such prediction sensitivity with respect to task-irrelevant variations manifests under both in-context learning and supervised fine-tuning, for both close-weight and open-weight general-purpose LLMs. Moreover, by examining the attention scores of an open-weight LLM, we discover a non-uniform attention pattern: training examples and variable names/values which happen to occupy certain positions in the prompt receive more attention when output tokens are generated, even though different positions are expected to receive roughly the same attention. This partially explains the sensitivity in the presence of task-irrelevant variations. We also consider a state-of-the-art tabular foundation model (TabPFN) trained specifically for data fitting. Despite being explicitly designed to achieve prediction robustness, TabPFN is still not immune to task-irrelevant variations. Overall, despite LLMs’ impressive predictive capabilities, currently they lack even the basic level of robustness to be used as a principled data-fitting tool. 1 Introduction Large language models (LLMs), as one of the most representative Generative AI applications, have been adopted across a wide variety of domains and tasks. While the majority of LLMs’ use cases are expectedly language-related (translation, Q&A, writing, ideation, etc.), there are also increasing uses of LLMs for tasks not apparently associated with language. We focus on one such practice, where LLM is used as an “off-the-shelf” data-fitting tool to generate predictions of interest. Data fitting refers to the task of using sets \{(\boldsymbol{X},Y)\} of numerical input values \boldsymbol{X} and target values Y to learn (fit) a model f(.) – i.e., Y\sim f(\boldsymbol{X}) – that can generate accurate predictions given inputs. For example, LLMs have been used for predicting property prices based on property features (e.g., number of bedrooms, bathrooms, etc.) and location information Tanlamai et al., 2024 ; performing time series forecasting and numeric prediction tasks in general by treating input data as a sequence of string tokens Gruver et al., 2023 ; Jin et al., 2023 ; Das et al., 2024 ; Goswami et al., 2024 ; Vacareanu et al., 2024 ; detecting anomalies in time series data Alnegheimish et al., 2024 ; Dong et al., 2024 ; Liu et al., 2025 ; generating synthetic data in data-hungry domains Seedat et al., 2023 ; Wang et al., 2024 ; Brand et al., 2023 ; Isomura et al., 2024 ; and support causal effect estimation by predicting counterfactual outcomes ( Huynh et al., 2025 ) . In all of these examples, LLMs essentially act as a drop-in replacement for traditional, tabular supervised learning techniques to generate predictions based on input data (See Appendix A for more details about these examples). A priori, it is unclear whether LLMs should be used for data fitting and predictions. On one hand, hailed as a general-purpose technology Eloundou et al., 2024 , there is evidence that LLMs’ impressive capabilities extend beyond language tasks and can perform data fitting quite well. Prior work has shown that LLMs, with just a few input-output examples included in the prompt (i.e., using few-shot prompting), can outperform many tabular supervised learning techniques for regression tasks ( Vacareanu et al., 2024 ) . Furthermore, when feature names are semantically meaningful, LLMs can perform feature selection automatically, which may further benefit data fitting ( Jeong et al., 2024 ) . On the other hand, LLMs vs. tabular supervised learning techniques perform “data fitting” and “prediction” in fundamentally different ways. Tabular supervised learning techniques perform data fitting by following well-defined algorithmic procedures that fit parametric functions (e.g., linear regression) or non-parametric functions (e.g., random forest) to training data. The trained models then produce predictions by directly applying the learned function (i.e., plug in the feature values of new input data). In contrast, the vast majority of LLMs today are based on the transformer architecture, which auto-regressively generates the next token based on prior tokens. With this architecture, LLMs perform data fitting by (i) having training examples in the prompt, namely in-context learning Brown et al., 2020 , or by (ii) supervised fine-tuning using the training data Hu et al., 2021 . 1 1 1 We do not consider the use case where LLMs generate and/or execute code to build tabular models. In such case, LLMs are being used as a coding tool rather than a data-fitting tool. They generate predictions one token at a time, where each token may correspond to only a segment of the prediction value. As an illustrative example, consider how an LLM generates a numerical prediction value. To generate a prediction value of 0.1875, OpenAI’s GPT-4o model needs to generate 4 tokens precisely in the following order: ‘‘0”, ‘‘.”, ‘‘187”, and ‘‘5”. 2 2 2 This example can be replicated at: https://platform.openai.com/tokenizer These fundamental differences raise the question whether LLMs provide a principled or robust way to fit numerical data and obtain predictions. In this work, we offer a critical assessment of using LLMs for data fitting. We construct a numeric prediction task and create synthetic data from scratch. Instead of using publicly available datasets, which LLMs may have been exposed to during training, using synthetic data enables a clean evaluation of LLM’s data-fitting capabilities. We experiment with general-purpose LLMs that are both close-weight (GPT-4o-mini) and open-weight (Llama-3-8B-instruct) as well as special-purpose tabular foundation models ( Hollmann et al., 2025 , TabPFN,) . Consistent with prior work Gruver et al., 2023 ; Vacareanu et al., 2024 , we find that LLMs perform reasonably well (especially after supervised fine-tuning), achieving performance that is competitive with commonly-used tabular supervised learning techniques. However, just because LLMs can be used for data fitting does not mean they should be. A principled data-fitting technique not only needs to achieve reasonable predictive performance, but also needs to possess at least a basic level of robustness so as not to be affected by factors that are fundamentally unrelated to the task itself. Our analyses show that LLMs have alarmingly poor prediction robustness. In particular, we apply several different types of task-irrelevant variations – such as altering variable names, variable (column) order, example (row) order, and data format – none of which has any effect on the underlying data-fitting task. While canonical tabular supervised learning techniques are completely unaffected (essentially by their design), LLMs’ predictions for the same data can change dramatically, causing the size of prediction error to sway as much as 82% in some settings. Further statistical tests confirm that such prediction sensitivity is indeed attributable to the task-irrelevant variations and cannot be explained by randomness in the LLM token

Read the complete text