Parallel Layer Normalization for Universal Approximation

Yunhao Ni 1 Yuxin Guo 1 Yuhe Liu 1 Wenxin Sun 1 Jie Luo 1 Wenjun Wu 1,2 Lei Huang 1,2 1 State Key Laboratory of Complex and Critical Software Environment (SKLCCSE), Beihang University, Beijing, China 2 Hangzhou International Innovation Institute, Beihang University, Hangzhou, China · 2025

Read Parallel Layer Normalization for Universal Approximation by Yunhao Ni 1 Yuxin Guo 1 Yuhe Liu 1 Wenxin Sun 1 Jie Luo 1 Wenjun Wu 1,2 Lei Huang 1,2 1 State Key Laboratory of Complex and Critical Software Environment (SKLCCSE), Beihang University, Beijing, China 2 Hangzhou International Innovation Institute, Beihang University, Hangzhou, China (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-13142.txt.

Description

Yunhao Ni and co-authors study how grouping normalization units changes what neural networks can express in a 2025 paper. Layer normalization is usually viewed as a training aid, but it also adds a joint nonlinearity between layers. The authors ask whether splitting it into parallel groups alters approximation power.

They prove a single shallow normalization unit can never yield universal approximation, no matter the width. Parallel groups, by contrast, become universal as group count grows. The result extends to deep nets, common variants, and convolutional designs under fit grouping. Experiments back the theory while showing group count alone does not rule behavior; placement and label structure matter too.

Catalog record

CREATORYunhao Ni 1 Yuxin Guo 1 Yuhe Liu 1 Wenxin Sun 1 Jie Luo 1 Wenjun Wu 1,2 Lei Huang 1,2 1 State Key Laboratory of Complex and Critical Software Environment (SKLCCSE), Beihang University, Beijing, China 2 Hangzhou International Innovation Institute, Beihang University, Hangzhou, China
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE103 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2505-13142

Browse the full library · About the Tower of Babel