AdaCap: An Adaptive Contrastive Approach for Small-Data Neural Networks
Bruno Belucci 1, Karim Lounici 2 and Katia Meziani 1 1- Université Paris Dauphine-PSL - CEREMADE Pl. du Maréchal de Lattre de Tassigny, 75016 Paris - France 2- Ecole Polytechnique - CMAP Route de Saclay, 91128 PALAISEAU - France · 2025
Read AdaCap: An Adaptive Contrastive Approach for Small-Data Neural Networks by Bruno Belucci 1, Karim Lounici 2 and Katia Meziani 1 1- Université Paris Dauphine-PSL - CEREMADE Pl. du Maréchal de Lattre de Tassigny, 75016 Paris - France 2- Ecole Polytechnique - CMAP Route de Saclay, 91128 PALAISEAU - France (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2511-20170.txt.
Description
Belucci, Lounici, and Meziani rescue neural networks on tiny tabular datasets. Their 2025 paper introduces AdaCap, an adaptive contrastive training scheme. Tree-based models dominate small-data tables stubbornly. Neural nets overfit or learn too little with few rows. AdaCap targets exactly that fragile regime.
A shuffle-based contrast loss shapes the learned codes. Shuffled features create contrast examples during training. A Tikhonov closed-form output mapping finishes the model. Ridge-style math replaces a learned final layer. Eighty-five real data tables tested the scheme. Multiple network designs were included in the trials. Gains were consistent and clearly real overall. Residual designs benefited most in small samples. A meta-predictor forecasts whether AdaCap will help. Dataset size, skew, and noise drive its predictions. It anticipates benefits accurately across the collection. Code and results are publicly available online. Small-data practitioners gain a targeted regularizer.
Catalog record
| CREATOR | Bruno Belucci 1, Karim Lounici 2 and Katia Meziani 1 1- Université Paris Dauphine-PSL - CEREMADE Pl. du Maréchal de Lattre de Tassigny, 75016 Paris - France 2- Ecole Polytechnique - CMAP Route de Saclay, 91128 PALAISEAU - France |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 13 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2511-20170