Learning curves theory for hierarchically compositional data with power-law distributed features
Francesco Cagnetta, Hyunmo Kang, Matthieu Wyart · 2025
Read Learning curves theory for hierarchically compositional data with power-law distributed features by Francesco Cagnetta, Hyunmo Kang, Matthieu Wyart (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-07067.txt.
Description
Francesco Cagnetta, Hyunmo Kang, and Matthieu Wyart study how learning machines improve with more data, in a 2025 theory paper. A known pattern, called a scaling law, says error falls in a steady curve as data grows. The authors ask how that curve arises when data is built in layers, the way grammar builds sentences from rules inside rules.
They study two tasks on such layered data: sorting items into classes and guessing the next word. For sorting, the curve's steepness is set by how the grammar rules are spread out, while the layered build inflates the error by a large fixed factor. For next-word guessing, the rule spread shapes only small wiggles in the curve, not its overall slope. The work ties two rival accounts of scaling laws into one picture.
Catalog record
| CREATOR | Francesco Cagnetta, Hyunmo Kang, Matthieu Wyart |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 53 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2505-07067