Three-Phase Transformer

Mohammad R. Abu Ayyash · 2026

Read Three-Phase Transformer by Mohammad R. Abu Ayyash (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-14430.txt.

Introduction

Abu Ayyash presents the Three-Phase Transformer, a structural prior for decoders that splits the hidden vector into equal cyclic channels on a SwiGLU, RMSNorm, RoPE, and GQA backbone. Each channel keeps its own normalization, a two-dimensional rotation between attention and feed-forward, and a rule tying GQA heads to the split. A DC subspace orthogonal to the channels carries a fixed Gabriel's horn profile as an absolute-position side channel alongside RoPE. At 123 million parameters, perplexity falls 7.20 percent against a matched baseline, adding only 1,536 parameters.

Catalog record

CREATORMohammad R. Abu Ayyash
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE121064
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2604.14430

Browse the full library · About the Tower of Babel