Optimizer Dynamics at the Edge of Stability with Differential Privacy
Ayana Hussain, Ricky Fang · 2025
Read Optimizer Dynamics at the Edge of Stability with Differential Privacy by Ayana Hussain, Ricky Fang (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2512-19019.txt.
Description
Ayana Hussain and Ricky Fang ask how privacy protection changes the way neural networks train. Differential privacy limits what a model can reveal about single training examples. It does so by clipping each example's gradient and adding random noise, which also reshapes the training path.
The pair compare plain gradient descent and Adam with their private versions. Private training tends to lower sharpness, a measure of how steep the loss surface is, and can stop models from reaching classic stability limits. Still, the familiar edge-of-stability patterns largely persist, and large step sizes with weak privacy can even pass those limits. Privacy, they conclude, makes training less predictable.
Catalog record
| CREATOR | Ayana Hussain, Ricky Fang |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 49 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2512-19019