Boosting Relational Deep Learning with Pretrained Tabular Models

Veronica Lachi ∗, Antonio Longa, Beatrice Bevilacqua, Bruno Lepri, Andrea Passerini, Bruno Ribeiro · 2025

Read Boosting Relational Deep Learning with Pretrained Tabular Models by Veronica Lachi ∗, Antonio Longa, Beatrice Bevilacqua, Bruno Lepri, Andrea Passerini, Bruno Ribeiro (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2504-04934.txt.

Introduction

Veronica Lachi, Antonio Longa, Beatrice Bevilacqua, Bruno Lepri, Andrea Passerini, and Bruno Ribeiro boost relational deep learning by letting graph networks lean on old-fashioned engineered features. Their paper from twenty twenty-five notes the trade: graph models read database structure natively but infer slowly, while flat tabular pipelines need hand-built features. LightRDL splits the labor. Engineered features carry the temporal story, so the graph can stay small and current.

The graph network then only hunts the relational patterns that feature engineering misses. On the RelBench benchmark, the hybrid lifts accuracy by up to a third over rivals. Inference runs hundreds of times faster than full graph models. Real-time prediction on live databases is the prize.

Catalog record

CREATORVeronica Lachi ∗, Antonio Longa, Beatrice Bevilacqua, Bruno Lepri, Andrea Passerini, Bruno Ribeiro
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE67 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2504-04934

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Boosting Relational Deep Learning with Pretrained Tabular Models Veronica Lachi ∗, Antonio Longa, Beatrice Bevilacqua, Bruno Lepri, Andrea Passerini, Bruno Ribeiro Abstract Relational databases, organized into tables connected by primary-foreign key relationships, are a common format for organizing data. Making predictions on relational data often involves transforming them into a flat tabular format through table joins and feature engineering, which serve as input to tabular methods. However, designing features that fully capture complex relational patterns remains challenging. Graph Neural Networks (GNNs) offer a compelling alternative by inherently modeling these relationships, but their time overhead during inference limits their applicability for real-time scenarios. In this work, we aim to bridge this gap by leveraging existing feature engineering efforts to enhance the efficiency of GNNs in relational databases. Specifically, we use GNNs to capture complex relationships within relational databases—patterns that are difficult to featurize, while employing engineered features to encode temporal information, thereby avoiding the need to retain the entire historical graph and enabling the use of smaller, more efficient graphs. Our LightRdl approach not only improves efficiency, but also outperforms existing models. Experimental results on the RelBench benchmark demonstrate that our framework achieves up to 33 % 33\% performance improvement and a 526 × 526\times inference speedup compared to GNNs, making it highly suitable for real-time inference. 1 Introduction Relational databases are extensively used in industry due to their flexibility, extensibility, and speed ( Berg et al., 2013 ; Johnson et al., 2016 ; Halpin & Morgan, 2010 ) . The information is organized into tables and records entities, their features, relations (via primary and foreign key relationships), and events, such as transactions with their associated timestamps. This format simplifies data maintenance and optimization while improving accessibility and retrieval through query languages like SQL ( Codd, 1970 ; Chamberlin & Boyce, 1974 ) . As a result, relational databases are integral to several applications, ranging from e-commerce platforms ( Agrawal et al., 2001 ) and social media networks ( Almabdy, 2018 ) to banking systems ( Aditya et al., 2002 ) and healthcare services ( Park et al., 2014 ) . Due to the presence of both timestepped events and relations, tasks over relational databases tend to be both temporal and relational, such as forecasting future product sales and predicting future customer purchases and churn ( Robinson et al., 2024 ) . For decades, companies have built in-house predictive models over these databases by flattening the complex temporal-relational data into tabular formats via meticulously engineered temporal and relational features ( Dong & Liu, 2018 ; Ganguli & Thakur, 2020 ) . These flattened data, often referred to as tabular data , are then fed as input to tabular models such as XGBoost and LightGBM ( Ke et al., 2017 ; Chen & Guestrin, 2016 ) . However, capturing all complex relational patterns in the data through feature engineering can be challenging ( Lam et al., 2017 ; Zheng & Casari, 2018 ) . A different paradigm proposes neural network models , such as Relational Graph Neural Networks (R-GNNs) ( Cvitkovic, 2020 ; Fey et al., 2024 ; Villaizán-Vallelado et al., 2024 ) , which model relational data directly as heterogeneous graphs. In this paradigm, each table row corresponds to an attributed node, and edges are defined through primary-foreign key relationships. R-GNNs eliminate the need for feature engineering ( Fey et al., 2024 ) and have demonstrated success in simplifying predictive models ( Robinson et al., 2024 ) . Figure 1: Overview of our proposed hybrid modeling framework LightRdl . The pipeline begins with feature-engineered tabular data processed by a tree-based model (e.g., LightGBM ). Knowledge distillation then generates embeddings summarizing temporal information, which are then utilized as additional node features for the static GNN responsible for the predictions. Despite the promise of R-GNNs, the strong performance and fast inference times of legacy tabular models — which we refer to as pretrained tabular models , that is, models trained on features engineered by domain experts — remains a challenge for the widespread adoption of R-GNNs. These pretrained models are lightweight, benefiting from powerful features derived through extensive feature engineering, and they achieve fast inference speed that R-GNNs often struggle to match. Additionally, while R-GNNs can offer improved performance in certain scenarios, the gains can be modest, and in most cases, their results align with those of traditional tabular models (as evidenced in Robinson et al. (2024) and Table 2 ). This diminishes the incentive for transitioning to R-GNNs, particularly when inference speed is a priority. This raises a key question: How can we combine the efficiency of pretrained tabular models with the relational modeling power of R-GNNs to achieve both high accuracy and fast inference? Specifically, can we leverage the feature engineering from pretrained tabular models to speed up R-GNNs while improving their performance? In this work, we propose a hybrid tabular-R-GNN modeling framework for predictive tasks on relational databases, which combines the relational modeling power of R-GNNs with the inference speed of tabular methods. Our approach uses the knowledge extracted from pretrained tabular models, distilled into MLPs. These MLP-generated embeddings are integrated as additional features into an R-GNN. Incorporating these features allows us to train the R-GNN on a smaller graph constructed only from the immediate time period preceding the inference step, rather than on the entire historical graph required by R-GNNs. This enables us to effectively leverage the temporal dynamics handled by the tabular models, significantly reducing inference times. An overview of our method, which we call LightRdl , is illustrated in Figure 1 . Our LightRdl not only enhances efficiency, but also outperforms existing models. Experimental results on the RelBench benchmark ( Robinson et al., 2024 ) demonstrate that our framework achieves up to 33\% performance improvement and 526\times inference speedup compared to R-GNNs, making it highly suitable for real-time inference. 2 Learning on Relational Databases Relational databases organize data across multiple interconnected tables, each containing entities with a shared schema. Relationships between tables define dependencies among entities. Formally: Definition 1 (Relational Database) . A relational database {\mathcal{R}} with N tables consists of a set of entities v\in{\mathcal{V}} , where each entity is uniquely indexed. The i -th table {\mathbb{T}}_{i} is defined as {\mathbb{T}}_{i}=\{{\bm{v}}_{v}\mid v\in{\mathcal{V}}:\phi(v)=i\} , where \phi:\mathcal{V}\rightarrow[N] maps entities to their corresponding tables, and each row {\bm{v}}_{v}=(p_{v},{\bm{k}}_{v},{\bm{x}}_{v},t_{v}) contains: • p_{v}\in{\mathbb{N}} : the primary key , uniquely identifying v ; • {\bm{k}}_{v}=(k_{v}^{1},\ldots,k_{v}^{N}) : foreign keys , k_{v}^{c}\in{\mathbb{N}} , linking to other tables, with k_{v}^{c}=0 if no link exists to table c ; • {\bm{x}}_{v}=(x_{v}^{1},\ldots,x_{v}^{d_{i}}) : features ; • t_{v} : the timestamp indicating when v appears. The set of all tables is {\mathcal{T}}=\{{\mathbb{T}}_{1},\ldots,{\mathbb{T}}_{N}\} . A link (i,j)\in{\mathcal{L}} exists between tables i and j if a foreign key in {\mathbb{T}}_{i} matches a primary key in {\mathbb{T}}_{j} , i.e., {\mathcal{L}}=\{(i,j)\mid\exists u\in{\mathbb{T}}_{i},v\in{\mathbb{T}}_{j},k^{j}_{u}=p_{v}\} . For instance, the rel-hm database from the RelBench benchmark ( Robinson et al., 2024 ) captures customer purchase histories on the H&M e-commerce platform. Further details about this dataset are provided in Appendix A . The dataset includes: (i) a customer table with attributes like gender and birth year; (ii) a product table with product details such as description and size; (iii) a transaction table recording customer purchases. Many

Read the complete text