Accelerating Large Language Models through Partially Linear Feed-Forward Network

Gansen Hu, Zhaoguo Wang, Jinglin Wei, Wei Huang, Haibo Chen Institute of Parallel and Distributed Systems, SEIEE, Shanghai Jiao Tong University · 2025

Read Accelerating Large Language Models through Partially Linear Feed-Forward Network by Gansen Hu, Zhaoguo Wang, Jinglin Wei, Wei Huang, Haibo Chen Institute of Parallel and Distributed Systems, SEIEE, Shanghai Jiao Tong University (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2501-10054.txt.

Description

Gansen Hu, Zhaoguo Wang, Jinglin Wei, Wei Huang, and Haibo Chen published this paper in 2025. It shrinks large language models by borrowing a compiler trick: constant folding, where predictable computations are precomputed away. The catch is that modern models use curved activation functions, which block the trick.

Their system, TARDIS, approximates those curves with straight lines over the input ranges that occur most often, letting whole chunks of the network fold away. Rare outlier inputs are caught by a small predictor and routed to the original computation. The result removes eighty percent of the parameters in the models' feed-forward layers. It beat leading pruning methods by up to sixty-five percent higher accuracy at the same size. Hooked into standard serving systems, a seven-billion-parameter model answered 1.4 to 1.6 times faster, at a modest accuracy cost. The authors pitch partial linearization as a kinder alternative to pruning.

Catalog record

CREATORGansen Hu, Zhaoguo Wang, Jinglin Wei, Wei Huang, Haibo Chen Institute of Parallel and Distributed Systems, SEIEE, Shanghai Jiao Tong University
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE57 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2501-10054

Browse the full library · About the Tower of Babel