On the Batch Size Selection in Stochastic Gradient Methods Using No-Replacement Sampling
Marco Boresta, Alberto De Santis, Stefano Lucidi · 2025
Read On the Batch Size Selection in Stochastic Gradient Methods Using No-Replacement Sampling by Marco Boresta, Alberto De Santis, Stefano Lucidi (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2506-08758.txt.
Description
Batch size in stochastic gradient methods should grow without replacement sampling, Marco Boresta, Alberto De Santis, and Stefano Lucidi argue in a 2025 paper. Recent methods base their efficiency and convergence on controlling the variance of the gradient batch estimate. That control is usually achieved by increasing the batch size dynamically as iterations proceed and the model nears a solution.
Statistical analyses in that literature often rely on sampling with replacement, which the authors call unrealistic in practice. They study batch selection based on sampling without replacement instead. The new selection still ensures global convergence. It also represents the variance reduction seen in practice more accurately, which leads to a smoother, more efficient batch update scheme for practitioners.
Catalog record
| CREATOR | Marco Boresta, Alberto De Santis, Stefano Lucidi |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 17 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2506-08758