How Do Transformers Model Physics? Investigating the Simple Harmonic Oscillator
Subhash Kantamneni, Ziming Liu, Max Tegmark, Jaroslaw Krzywanski, Marcin Sosnowski, Karolina Grabowska, Dorian Skrobek, Ghulam Moeen Uddin · 2024
Read How Do Transformers Model Physics? Investigating the Simple Harmonic Oscillator by Subhash Kantamneni, Ziming Liu, Max Tegmark, Jaroslaw Krzywanski, Marcin Sosnowski, Karolina Grabowska, Dorian Skrobek, Ghulam Moeen Uddin (2024) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/pmc-11592621.txt.
Description
Subhash Kantamneni and co-authors probe how transformer models handle a basic physics system, the simple harmonic oscillator. Their 2024 study asks whether the models use methods people can recognize, or invent an alien physics of their own. The oscillator, a mass on a spring with damping, has exact known solutions for comparison.
The team set four tests for whether a candidate method truly runs inside the model. These ranged from reading the method's halfway values out of hidden states to changing those states and watching predictions shift. All signs pointed to the matrix exponential method, a standard numerical approach. The authors offer their test kit for wider use on harder systems.
Catalog record
| CREATOR | Subhash Kantamneni, Ziming Liu, Max Tegmark, Jaroslaw Krzywanski, Marcin Sosnowski, Karolina Grabowska, Dorian Skrobek, Ghulam Moeen Uddin |
|---|---|
| YEAR | 2024 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 46 KB |
| VERSION | 1.0 |
| LICENSE | CC BY 4.0 |
| RIGHTS | CC BY |
| SOURCE | Europe PMC |