Multitask-Informed Prior for In-Context Learning on Tabular Data: Application to Steel Property Prediction

Dimitrios Sinodinos, Bahareh Nikpour, Jack Y. Wei, Sushant Sinha, Xiaoping Ma, Kashif Rehman, Stephen Yue, Narges Armanfard · 2026

Read Multitask-Informed Prior for In-Context Learning on Tabular Data: Application to Steel Property Prediction by Dimitrios Sinodinos, Bahareh Nikpour, Jack Y. Wei, Sushant Sinha, Xiaoping Ma, Kashif Rehman, Stephen Yue, Narges Armanfard (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-22738.txt.

Introduction

Sinodinos and co-authors adapt a tabular foundation model to predict steel properties. The target is hot rolling, where chemistry, process settings, and microstructure interact. Their method gives the TabPFN model a multitask-aware prior. One route averages targets into a single signal. A second route adds task-specific adapters in fine-tuning. Tests use an industrial thin slab direct rolling dataset. The multitask versions beat classic methods and recent tabular models on several metrics. They also beat task-specific fine-tuning on speed and accuracy.

Catalog record

CREATORDimitrios Sinodinos, Bahareh Nikpour, Jack Y. Wei, Sushant Sinha, Xiaoping Ma, Kashif Rehman, Stephen Yue, Narges Armanfard
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE53651
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2603.22738

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Multitask-Informed Prior for In-Context Learning on Tabular Data: Application to Steel Property Prediction Dimitrios Sinodinos, Bahareh Nikpour, Jack Y. Wei, Sushant Sinha, Xiaoping Ma, Kashif Rehman, Stephen Yue, Narges Armanfard Abstract Accurate prediction of mechanical properties of steel during hot rolling processes, such as Thin Slab Direct Rolling (TSDR), remains challenging due to complex interactions among chemical compositions, processing parameters, and resultant microstructures. Traditional empirical and experimental methodologies, while effective, are often resource-intensive and lack adaptability to varied production conditions. Moreover, most existing approaches do not explicitly leverage the strong correlations among key mechanical properties, missing an opportunity to improve predictive accuracy through multitask learning. To address this, we present a multitask learning framework that injects multitask awareness into the prior of TabPFN—a transformer-based foundation model for in-context learning on tabular data—through novel fine-tuning strategies. Originally designed for single-target regression or classification, we augment TabPFN’s prior with two complementary approaches: (i) target averaging, which provides a unified scalar signal compatible with TabPFN’s single-target architecture, and (ii) task-specific adapters, which introduce task-specific supervision during fine-tuning. These strategies jointly guide the model toward a multitask-informed prior that captures cross-property relationships among key mechanical metrics. Extensive experiments on an industrial TSDR dataset demonstrate that our multitask adaptations outperform classical machine learning methods and recent state-of-the-art tabular learning models across multiple evaluation metrics. Notably, our approach enhances both predictive accuracy and computational efficiency compared to task-specific fine-tuning, demonstrating that multitask-aware prior adaptation enables foundation models for tabular data to deliver scalable, rapid, and reliable deployment for automated industrial quality control and process optimization in TSDR. Fig. 1 : Schematic of Direct Strip Processing Complex I Introduction Steel is used widely across industries, including automotive , construction , oil and gas , and in various forms such as sheets, rods, and wires. The most critical step in forming steel into these shapes is the hot rolling (HR) process which is designed to both reduce the thickness of steel slabs and enhance their mechanical properties [ 18 ] . Thin Slab Direct Rolling (TSDR) is a more advanced version of hot strip rolling that focuses on better productivity and energy efficiency [ 12 ] . The TSDR process starts by casting liquid steel into thin slabs about 70-80 mm thick. Right after casting, these slabs go into a tunnel furnace for a short heating of 15-20 minutes to ensure temperature homogeneity. Hence, the slab’s width is adjusted with an edger, and then it is sent to the roughing stand. Here, the slab undergoes drastic reduction to form a transfer bar, about 35-40 mm thick. This bar moves on to the finishing stands where it’s rolled in several subsequent stages to reach the desired thickness strip. Figure 1 shows an overview of this process. In the high-speed, high-temperature environment of a TSDR mill, process control is predominantly established before the rolling begins. To a process control engineer, the primary objective is to produce steel that not only meets specific thickness requirements but also achieves the desired mechanical properties, all while operating within the constraints of the rolling mill. The desired thickness of the HR steel is determined by the reduction settings at each rolling stand, which are configured before the start of the rolling process. The mechanical properties of the steel, on the other hand, depend on its chemical composition and the final microstructure, which is shaped by thermomechanical controlled processing (TMCP). TMCP involves careful adjustment of rolling speeds, reductions across different passes, and rolling temperatures, parameters that are crucial for tailoring the properties to meet specific demands. Given the complexity of the TSDR process, process control engineers must rely on predictive models that can estimate critical mechanical properties in advance. These models enable fine-tuning of the rolling process and chemical composition, ensuring efficient, safe, and specification-compliant production while minimizing the need for costly post-production adjustments and testing. Accurate predictions require a robust understanding of thermomechanical processes and the intricate relationships between the material’s chemical composition, microstructure, and processing parameters, which are essential for optimizing production and quality control. Early machine learning approaches to steel property prediction primarily relied on classical models such as support vector machines and gradient-boosted decision trees [ 4 ] . While effective in many scenarios, these methods struggle to capture complex feature dependencies and fail to generalize well in high-dimensional tabular datasets. More recently, deep learning models, such as feedforward neural networks (FNNs), recurrent architectures (RNNs, LSTMs), and convolutional neural networks (CNNs), have been explored for steel property prediction. Although these methods aim to extract meaningful representations from chemical and process data to enhance prediction accuracy, single-task deep learning models often struggle to capture the interdependencies between multiple mechanical properties, limiting their generalization in dynamic industrial settings. To address this limitation, multi-task learning (MTL) [ 3 ] has been increasingly adopted as a strategy to improve efficiency and generalization by jointly modeling multiple related tasks. MTL approaches range from hard parameter sharing, where a common feature representation is learned for all tasks, to soft parameter sharing, which allows task-specific representations while enabling knowledge transfer [ 22 ] . In addition to improving predictive performance, MTL offers notable computational benefits—particularly in scenarios involving limited data or costly training resources—by reducing the number of independent models that need to be trained and maintained [ 23 ] . This shared architecture results in faster inference times and reduced memory usage, making MTL especially advantageous for real-time or large-scale industrial deployment. Recently, Tabular Prior-data Fitted Network (TabPFN) [ 7 ] has achieved state-of-the-art performance on small tabular datasets by using in-context learning (ICL), where a model adapts to new tasks directly from examples in its input context without requiring parameter updates. As a transformer-based Prior-Data Fitted Network (PFN), TabPFN is trained on synthetic datasets to approximate Bayesian posterior inference in a single forward pass, enabling rapid generalization across tasks. It represents a step toward foundation models for tabular data—models pre-trained to transfer broadly to downstream applications with minimal supervision. Recent work has shown that fine-tuning TabPFN’s prior on domain-specific datasets can further improve inference-time performance over a fully general prior [ 7 ] . While effective in single-task contexts, in MTL, this approach requires training a separate model for each target, which creates redundancy in compute and storage, and fails to exploit cross-task information that could improve generalization. Moreover, because TabPFN is pre-trained to predict a single target variable, performing multitask pre-training would require redesigning the output interface and generating new synthetic datasets that model joint target distributions—a major theoretical and computational undertaking. As a simpler and more practical alternative to predicting multiple continuous-valued mechanical properties, we propose injecting multitask awareness during fine-tuning by encoding the targets into a single signal, thereby preserving the off-the-shelf usability of pre-trained TabPFN models. We explore two strategies for implementing this idea: (1) fine-tuning on the average of all task targets, which provides a simple scalar proxy signal compatible with TabPFN’s single-target interface, and (2) incorporating a lightweight multitask adapter that introduces task-specific predictions during fine-tuning. It has also been shown with TabPFN [ 7 ] that fine-tuning on related but not identical signals can outperform zero-shot inference, suggesting that proxy objectives can yield stronger priors. Building on this insight,

Read the complete text