Optimizer Dynamics at the Edge of Stability with Differential Privacy

Ayana Hussain, Ricky Fang · 2025

Read Optimizer Dynamics at the Edge of Stability with Differential Privacy by Ayana Hussain, Ricky Fang (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2512-19019.txt.

Description

Ayana Hussain and Ricky Fang ask how privacy protection changes the way neural networks train. Differential privacy limits what a model can reveal about single training examples. It does so by clipping each example's gradient and adding random noise, which also reshapes the training path.

The pair compare plain gradient descent and Adam with their private versions. Private training tends to lower sharpness, a measure of how steep the loss surface is, and can stop models from reaching classic stability limits. Still, the familiar edge-of-stability patterns largely persist, and large step sizes with weak privacy can even pass those limits. Privacy, they conclude, makes training less predictable.

Catalog record

CREATORAyana Hussain, Ricky Fang
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE49 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2512-19019

Browse the full library · About the Tower of Babel