PLDR-LLMs learn a generalizable tensor operator that can replace its own deep neural net at inference
Burc Gokden · 2025
Read PLDR-LLMs learn a generalizable tensor operator that can replace its own deep neural net at inference by Burc Gokden (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2502-13502.txt.
Description
Burc Gokden reports a striking property of a language model family built on power law decoder representations. Once trained, the model's reasoned outputs act like fixed tensors that barely change, so a stored copy of one key tensor can stand in for the deep network itself at run time. A cache for that tensor, added beside the standard key-value cache, then speeds up inference.
The match is close: cached outputs keep the same error scores and test results, agreeing to many decimal places. Tests that swap in borrowed, random, or identity tensors behave very differently, and the standard attention method turns out to be a special case of this design. The author outlines the shared traits of the learned outputs and shares a training and inference setup that uses both caches.
Catalog record
| CREATOR | Burc Gokden |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 53 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2502-13502