Approximation of Permutation Invariant Polynomials by Transformers: Efficient Construction in Column-Size
Naoki Takeshita † , Masaaki Imaizumi †‡ · 2025
Read Approximation of Permutation Invariant Polynomials by Transformers: Efficient Construction in Column-Size by Naoki Takeshita † , Masaaki Imaizumi †‡ (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2502-11467.txt.
Description
Naoki Takeshita and Masaaki Imaizumi ask how well transformer networks can copy a special kind of formula, in a 2025 theory paper. Transformers are the networks behind most modern language tools. The formulas here take tables of numbers as input and give the same answer when columns are swapped. Such symmetry shows up in many data tasks.
The authors build transformers that copy these formulas and track how network size affects the error. Their proof uses traits of symmetric formulas, a classic topic in algebra. The link they find is explicit, tying size to accuracy in a clear way. Results like this help explain why these networks are so flexible in practice.
Catalog record
| CREATOR | Naoki Takeshita † , Masaaki Imaizumi †‡ |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 71 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2502-11467