Parallel Layer Normalization for Universal Approximation
Yunhao Ni 1 Yuxin Guo 1 Yuhe Liu 1 Wenxin Sun 1 Jie Luo 1 Wenjun Wu 1,2 Lei Huang 1,2 1 State Key Laboratory of Complex and Critical Software Environment (SKLCCSE), Beihang University, Beijing, China 2 Hangzhou International Innovation Institute, Beihang University, Hangzhou, China · 2025
Read Parallel Layer Normalization for Universal Approximation by Yunhao Ni 1 Yuxin Guo 1 Yuhe Liu 1 Wenxin Sun 1 Jie Luo 1 Wenjun Wu 1,2 Lei Huang 1,2 1 State Key Laboratory of Complex and Critical Software Environment (SKLCCSE), Beihang University, Beijing, China 2 Hangzhou International Innovation Institute, Beihang University, Hangzhou, China (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-13142.txt.
Description
Yunhao Ni and co-authors study how grouping normalization units changes what neural networks can express in a 2025 paper. Layer normalization is usually viewed as a training aid, but it also adds a joint nonlinearity between layers. The authors ask whether splitting it into parallel groups alters approximation power.
They prove a single shallow normalization unit can never yield universal approximation, no matter the width. Parallel groups, by contrast, become universal as group count grows. The result extends to deep nets, common variants, and convolutional designs under fit grouping. Experiments back the theory while showing group count alone does not rule behavior; placement and label structure matter too.
Catalog record
| CREATOR | Yunhao Ni 1 Yuxin Guo 1 Yuhe Liu 1 Wenxin Sun 1 Jie Luo 1 Wenjun Wu 1,2 Lei Huang 1,2 1 State Key Laboratory of Complex and Critical Software Environment (SKLCCSE), Beihang University, Beijing, China 2 Hangzhou International Innovation Institute, Beihang University, Hangzhou, China |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 103 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2505-13142