Three-Phase Transformer
Mohammad R. Abu Ayyash · 2026
Read Three-Phase Transformer by Mohammad R. Abu Ayyash (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-14430.txt.
Introduction
Abu Ayyash presents the Three-Phase Transformer, a structural prior for decoders that splits the hidden vector into equal cyclic channels on a SwiGLU, RMSNorm, RoPE, and GQA backbone. Each channel keeps its own normalization, a two-dimensional rotation between attention and feed-forward, and a rule tying GQA heads to the split. A DC subspace orthogonal to the channels carries a fixed Gabriel's horn profile as an absolute-position side channel alongside RoPE. At 123 million parameters, perplexity falls 7.20 percent against a matched baseline, adding only 1,536 parameters.
Catalog record
| CREATOR | Mohammad R. Abu Ayyash |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 121064 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2604.14430