Grokking of Diffusion Models: Case Study on Modular Addition
Joon Hyeok Kim Yonghyun Park Mattis Dalsætra Østby Jiatao GuDepartment of Computer and Information ScienceUniversity of PennsylvaniaPhiladelphia, PA 19104, USA {hozy, park19, mdals, jgu32}@seas.upenn.edu · 2026
Read Grokking of Diffusion Models: Case Study on Modular Addition by Joon Hyeok Kim Yonghyun Park Mattis Dalsætra Østby Jiatao GuDepartment of Computer and Information ScienceUniversity of PennsylvaniaPhiladelphia, PA 19104, USA {hozy, park19, mdals, jgu32}@seas.upenn.edu (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-17673.txt.
Description
Kim, Park, Østby, and Gu give a careful account of diffusion-model learning, using grokking in an image-based modular addition setting as a testbed. In a single-image regime, the model encodes inputs as cyclic codes and joins them at the attention-feedforward interface, finding trig identities. In a varied-image regime with high variety, step-by-step sampling splits the task into math and visual polish phases, with the switch tracked by an internal frequency-entropy signal. Overall, the study explains diffusion-model learning from inside.
Catalog record
| CREATOR | Joon Hyeok Kim Yonghyun Park Mattis Dalsætra Østby Jiatao GuDepartment of Computer and Information ScienceUniversity of PennsylvaniaPhiladelphia, PA 19104, USA {hozy, park19, mdals, jgu32}@seas.upenn.edu |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 52350 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2604.17673