Multi-Gate Residuals

Zhizhan Zheng Feiyun Zhang Shuchun Liu Tian Xia Xi Liu Dasheng Hu Hongquan Zhou · 2026

Read Multi-Gate Residuals by Zhizhan Zheng Feiyun Zhang Shuchun Liu Tian Xia Xi Liu Dasheng Hu Hongquan Zhou (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-23259.txt.

Description

Zheng and six coauthors propose Multi-Gate Residuals, which steady signal scales without added message cost. Prior gaps help address wild signal growth across deep residual layers but incur big message cost. Multi-Gate Residuals use a simple scoring and gating setup to maintain multi-stream context, paired with Attention Pooling for extracting hidden states from stream states. Tests show the method is usable for large-scale training and use, giving real score gains over old designs.

Catalog record

CREATORZhizhan Zheng Feiyun Zhang Shuchun Liu Tian Xia Xi Liu Dasheng Hu Hongquan Zhou
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE42143
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2605.23259

Browse the full library · About the Tower of Babel