KnowsTFM: Knowledge-Informed Fine-Tuning of Small Tabular Foundation Models

Boshko Koloski Xiangjian Jiang Senja Pollak, Blaž Škrlj Mateja Jamnik Nikola Simidjievski · 2026

Read KnowsTFM: Knowledge-Informed Fine-Tuning of Small Tabular Foundation Models by Boshko Koloski Xiangjian Jiang Senja Pollak, Blaž Škrlj Mateja Jamnik Nikola Simidjievski (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2606-30258.txt.

Description

Boshko Koloski, Xiangjian Jiang, Senja Pollak, Blaž Škrlj, Mateja Jamnik, and Nikola Simidjievski steer small tabular foundation models with curated domain knowledge. Such models predict from tables after pretraining on synthetic tasks. In niche domains, data are scarce, high-dimensional, and distributionally shifted. Carefully built specialist methods can still beat the general models there. Knowledge graphs from those domains sit unused by default.

Their fine-tuning recipe, KnowsTFM, injects that relational knowledge during adaptation. Structural attention priors are derived directly from the knowledge graphs. Parameter-efficient low-rank updates carry the rest of the fine-tuning. Nanoscale variants of two tabular model families serve as testbeds. Injecting structural knowledge yields meaningful gains in specialist settings. Gains on general-domain tasks, by contrast, are only marginal. Continual fine-tuning of frontier models can collapse pretrained knowledge, they warn.

Catalog record

CREATORBoshko Koloski Xiangjian Jiang Senja Pollak, Blaž Škrlj Mateja Jamnik Nikola Simidjievski
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE83 KB
LICENSEhttp://creativecommons.org/licenses/by-sa/4.0/
RIGHTSCC BY SA
SOURCEarXiv

Source: https://arxiv.org/abs/2606.30258

Browse the full library · About the Tower of Babel