On the Batch Size Selection in Stochastic Gradient Methods Using No-Replacement Sampling

Marco Boresta, Alberto De Santis, Stefano Lucidi · 2025

Read On the Batch Size Selection in Stochastic Gradient Methods Using No-Replacement Sampling by Marco Boresta, Alberto De Santis, Stefano Lucidi (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2506-08758.txt.

Description

Batch size in stochastic gradient methods should grow without replacement sampling, Marco Boresta, Alberto De Santis, and Stefano Lucidi argue in a 2025 paper. Recent methods base their efficiency and convergence on controlling the variance of the gradient batch estimate. That control is usually achieved by increasing the batch size dynamically as iterations proceed and the model nears a solution.

Statistical analyses in that literature often rely on sampling with replacement, which the authors call unrealistic in practice. They study batch selection based on sampling without replacement instead. The new selection still ensures global convergence. It also represents the variance reduction seen in practice more accurately, which leads to a smoother, more efficient batch update scheme for practitioners.

Catalog record

CREATORMarco Boresta, Alberto De Santis, Stefano Lucidi
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE17 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2506-08758

Browse the full library · About the Tower of Babel