How good is PAC-Bayes at explaining generalisation?
Antoine Picard-Weibel, Eugenio Clerico, Roman Moscoviz, Benjamin Guedj · 2025
Read How good is PAC-Bayes at explaining generalisation? by Antoine Picard-Weibel, Eugenio Clerico, Roman Moscoviz, Benjamin Guedj (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2503-08231.txt.
Description
Antoine Picard-Weibel and co-authors ask how well a PAC-Bayes bound can explain generalization in a 2025 paper. Such bounds promise guarantees that a learned model will do well on new data. The authors derive conditions a bound must meet to give a meaningful guarantee. Their analysis shows the best possible guarantee hinges only on how the starting distribution spreads risk.
In short, reaching a target level needs the starting choice to place enough weight on strong predictors. That finding bears on the common practice of building the starting choice from data in deep learning. It also questions claims that the framework by itself explains why models generalize. Readers get a clear limit on what these bounds can say.
Catalog record
| CREATOR | Antoine Picard-Weibel, Eugenio Clerico, Roman Moscoviz, Benjamin Guedj |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 71 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2503-08231