Approximation of Permutation Invariant Polynomials by Transformers: Efficient Construction in Column-Size

Naoki Takeshita † , Masaaki Imaizumi †‡ · 2025

Read Approximation of Permutation Invariant Polynomials by Transformers: Efficient Construction in Column-Size by Naoki Takeshita † , Masaaki Imaizumi †‡ (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2502-11467.txt.

Description

Naoki Takeshita and Masaaki Imaizumi ask how well transformer networks can copy a special kind of formula, in a 2025 theory paper. Transformers are the networks behind most modern language tools. The formulas here take tables of numbers as input and give the same answer when columns are swapped. Such symmetry shows up in many data tasks.

The authors build transformers that copy these formulas and track how network size affects the error. Their proof uses traits of symmetric formulas, a classic topic in algebra. The link they find is explicit, tying size to accuracy in a clear way. Results like this help explain why these networks are so flexible in practice.

Catalog record

CREATORNaoki Takeshita † , Masaaki Imaizumi †‡
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE71 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2502-11467

Browse the full library · About the Tower of Babel