AdaCap: An Adaptive Contrastive Approach for Small-Data Neural Networks

Bruno Belucci 1, Karim Lounici 2 and Katia Meziani 1 1- Université Paris Dauphine-PSL - CEREMADE Pl. du Maréchal de Lattre de Tassigny, 75016 Paris - France 2- Ecole Polytechnique - CMAP Route de Saclay, 91128 PALAISEAU - France · 2025

Read AdaCap: An Adaptive Contrastive Approach for Small-Data Neural Networks by Bruno Belucci 1, Karim Lounici 2 and Katia Meziani 1 1- Université Paris Dauphine-PSL - CEREMADE Pl. du Maréchal de Lattre de Tassigny, 75016 Paris - France 2- Ecole Polytechnique - CMAP Route de Saclay, 91128 PALAISEAU - France (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2511-20170.txt.

Description

Belucci, Lounici, and Meziani rescue neural networks on tiny tabular datasets. Their 2025 paper introduces AdaCap, an adaptive contrastive training scheme. Tree-based models dominate small-data tables stubbornly. Neural nets overfit or learn too little with few rows. AdaCap targets exactly that fragile regime.

A shuffle-based contrast loss shapes the learned codes. Shuffled features create contrast examples during training. A Tikhonov closed-form output mapping finishes the model. Ridge-style math replaces a learned final layer. Eighty-five real data tables tested the scheme. Multiple network designs were included in the trials. Gains were consistent and clearly real overall. Residual designs benefited most in small samples. A meta-predictor forecasts whether AdaCap will help. Dataset size, skew, and noise drive its predictions. It anticipates benefits accurately across the collection. Code and results are publicly available online. Small-data practitioners gain a targeted regularizer.

Catalog record

CREATORBruno Belucci 1, Karim Lounici 2 and Katia Meziani 1 1- Université Paris Dauphine-PSL - CEREMADE Pl. du Maréchal de Lattre de Tassigny, 75016 Paris - France 2- Ecole Polytechnique - CMAP Route de Saclay, 91128 PALAISEAU - France
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE13 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2511-20170

Browse the full library · About the Tower of Babel