Switchable Activation Networks

Laha Ale, Ning Zhang, Scott A. King, Pingzhi Fan · 2026

Read Switchable Activation Networks by Laha Ale, Ning Zhang, Scott A. King, Pingzhi Fan (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-06601.txt.

Description

Ale and team bring SWAN, a tool giving each neural unit with an input-dependent binary gate, learning when each unit should be active or inactive for work allocation. Deep networks and large generative models achieve performance yet carry cost blocking resource-tight deployment. Dropout steadies training, while pruning and low-rank splitting turn models post hoc into static forms with flex. Unlike pruning, SWAN learns context-set activation patterns backing efficient dynamic inference and switch to compact dense models, joining sparsity, pruning, and adaptive use in one plan.

Catalog record

CREATORLaha Ale, Ning Zhang, Scott A. King, Pingzhi Fan
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE41885
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2603.06601

Browse the full library · About the Tower of Babel