Lecture notes: Stochastic gradient methods

2026

Read Lecture notes: Stochastic gradient methods (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2606-03953.txt.

Introduction

Weissmann wrote lecture notes that open a path into stochastic gradient methods. The notes start with plain gradient methods for unconstrained problems. They study how those methods converge under set conditions. The conditions include smoothness, convexity, and strong convexity. A Polyak-Łojasiewicz condition also appears in that survey.

The notes then turn to stochastic approximation and stochastic gradient descent. Machine learning risk, both sampled and expected, drives that shift. Core focus stays on convergence theory throughout. Readers get almost sure limits and limits in expectation. Classic rates are derived along the way. Later parts treat faster almost sure rates and variance reduction methods.

Catalog record

YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE284 KB
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2606.03953

Browse the full library · About the Tower of Babel