Algorithmic Recourse of In-Context Learning for Tabular Data

Wenshuo Dong, Jiaming Zhang, Shaopneg Fu, Hongbin Lin, Di Wang, Lijie Hu · 2026

Read Algorithmic Recourse of In-Context Learning for Tabular Data by Wenshuo Dong, Jiaming Zhang, Shaopneg Fu, Hongbin Lin, Di Wang, Lijie Hu (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-31272.txt.

Introduction

Dong and colleagues study algorithmic recourse for tabular decisions made by in-context learning. As predictive models spread in high-stakes settings like credit approval, affected people need actionable routes to a better outcome, yet recourse under in-context learning was largely unexplored. The authors prove recourse stays well-defined and bounded and converges to classical solutions as context grows. Their adaptive subspace method generates sparse, actionable recourse for black-box models with fewer queries, matching existing quality across datasets.

Catalog record

CREATORWenshuo Dong, Jiaming Zhang, Shaopneg Fu, Hongbin Lin, Di Wang, Lijie Hu
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE77612
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2605.31272

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Algorithmic Recourse of In-Context Learning for Tabular Data Wenshuo Dong, Jiaming Zhang, Shaopneg Fu, Hongbin Lin, Di Wang, Lijie Hu Abstract As predictive models are increasingly deployed in high-stakes settings such as credit approval, there is a growing need for post-hoc methods that provide recourse to affected individuals. Many such models operate on tabular data, where features correspond to real-world attributes. Recently, in-context learning (ICL) has enabled large language models to perform tabular prediction by conditioning on labeled examples at inference time, without explicit training. However, algorithmic recourse for tabular decision-making under ICL remains largely unexplored. In this work, we present the first study of algorithmic recourse for tabular data under ICL. We carry out a theoretical analysis, showing that recourse remains well-defined and bounded, and we characterize how recourse converges toward classical solutions as the context size increases. In practice, we propose a novel zeroth-order recourse framework, Adaptive Subspace Recourse for In-Context Learning (ASR-ICL), that efficiently generates actionable and sparse recourse for black-box ICL models. The proposed framework naturally extends to multi-class tabular tasks. Experiments across multiple real-world datasets and models demonstrate that ASR-ICL achieves recourse quality comparable to existing methods with fewer queries and empirically confirm the predicted convergence behavior, supporting our theoretical analysis. † † affiliationnotice: Equal contribution 1 Introduction Machine learning models are increasingly deployed in high-stakes decision-making domains such as credit lending and criminal justice ( Barocas & Selbst 2016 ) . Consequently, there is a growing emphasis on providing algorithmic recourse to individuals who are adversely impacted by predictions, offering actionable guidance on how to obtain a favorable outcome ( Voigt & Von dem Bussche 2017 ) . For example, when a loan application is rejected by an automated decision system, algorithmic recourse can suggest feasible actions, such as improving income stability or reducing outstanding debt, that may lead to a different outcome. Algorithmic recourse is an explanation framework that provides individuals with actionable and feasible feature changes that could lead to a favorable decision outcome ( Ustun et al. 2019 ) . Unlike post-hoc explanations that describe why a decision was made, recourse focuses on actionable changes that an individual can implement to obtain a different outcome ( Wachter et al. 2017 ) . As a result, algorithmic recourse has become a central component of responsible and trustworthy machine learning. Many high-stakes decision systems operate on tabular data, where features correspond to semantically meaningful and potentially actionable real-world attributes such as income, employment status, or medical indicators ( Kleinberg et al. 2018 ; Pawelczyk et al. 2020 ) . This makes tabular prediction an important setting for algorithmic recourse since recourse feature changes can often be interpreted as feasible actions. Recently, in-context learning (ICL) with large language models (LLMs) has emerged as a powerful paradigm for tabular prediction ( Wen et al. 2025 ) . Models such as TabPFN and TabICL demonstrate that strong tabular predictors can be obtained without explicit training by conditioning on a set of labeled examples provided at inference time ( Hollmann et al. 2025 ; Qu et al. 2025 ) . In this paradigm, predictions are dynamically induced from the context rather than produced by a fixed trained model. As a result, the effective decision rule varies across contexts, rendering the predictor inherently black-box and input-dependent. Many prior works have successfully applied algorithmic recourse to trained predictive models using various optimization techniques, such as gradient-based methods, integer programming, and causal modeling ( Karimi et al. 2021 ; Ustun et al. 2019 ) . These approaches typically assume a fixed decision function obtained after training, under which recourse is defined and computed ( Mothilal et al. 2020 ; Ustun et al. 2019 ; Poyiadzi et al. 2020 ; Pawelczyk et al. 2020 ) . In contrast, in-context learning induces predictions dynamically from the provided context; thus, the effective decision rule can vary across contexts even for the same individual ( Brown et al. 2020 ) . The effective models faced by the user is context-conditioned: changing the context can change both the prediction and the recourse action, even for the same query instance. Therefore, recourse under ICL raises new questions that do not arise in the classical fixed-model setting, including whether recourse is well-defined for a given context, whether it remains stable across contexts, and whether it can be computed efficiently with only black-box access. Despite the growing adoption of ICL for tabular prediction, the implications of such dynamic decision rules for algorithmic recourse remain largely unexplored. In this work, we present the first systematic study of algorithmic recourse under in-context learning for tabular data, addressing a critical gap between recent advances in tabular ICL and the absence of recourse in such settings. On the theoretical side, we formalize recourse with respect to context-conditioned predictors and establish that recourse remains well-defined and admits feasibility and boundedness guarantees. In particular, for a linear regression task trained via a single-layer linear self-attention model, we show that the cost of optimal recourse can be upper bounded and that these bounds tighten as the number of context examples increases, revealing a convergence of ICL-based recourse toward classical recourse for trained models. In practice, we further propose Adaptive Subspace Recourse for In-Context Learning (ASR-ICL), an efficient zeroth-order recourse framework for black-box ICL models. ASR-ICL adaptively concentrates the search on a small subset of important features, enabling efficient recourse computation in high-dimensional tabular spaces without access to gradients or model internals. Importantly, ASR-ICL naturally extends beyond binary classification and supports multi-class tabular decision tasks without task-specific modifications. Extensive experiments on multiple real-world datasets and across a range of tabular ICL models and general-purpose language models show that ASR-ICL achieves recourse validity comparable to existing methods for trained models while consistently yielding lower recourse costs. We further empirically confirm the theoretical convergence behavior as the number of context instances increases and demonstrate that ASR-ICL naturally extends to multi-class tabular decision tasks. Our main contributions are summarized as follows: • We present the first theoretical analysis of algorithmic recourse under in-context learning, establishing that recourse remains well-defined and bounded despite the dynamic, context-conditioned nature of ICL. We further show that, as the number of context examples increases, ICL-based recourse converges to classical linear-model recourse with provable perturbation bounds. • In practice, we propose ASR-ICL, the first recourse framework specifically designed for black-box ICL models, which enables efficient and sparse recourse in high-dimensional spaces under expensive model queries via adaptive zeroth-order optimization. • Extensive experiments on diverse real-world datasets and across multiple models demonstrate that ASR-ICL achieves recourse validity and cost comparable to existing methods for trained models, while naturally supporting multi-class tabular decision tasks. In particular, our results confirm the predicted stabilization of recourse behavior as the number of context instances increases. 2 Related Work Algorithmic Recourse. Prior works have studied algorithmic recourse for predictive models, aiming to generate counterfactual explanations that suggest actionable changes to reverse unfavorable decisions, primarily in binary classification settings ( Karimi et al. 2020 ; De Toni et al. 2025 ; Gao & Lakkaraju 2023 ; Kanamori et al. 2024 ; Ross et al. 2021 ) . Beyond classification, CARAT ( Datta et al. 2022 ) extends recourse to unsupervised anomaly detection

Read the complete text