FFN Fusion: Rethinking Sequential Computation in Large Language Models
Akhiad Bercovich *, Mohammad Dabbah *, Omri Puny *, Ido Galil, Amnon Geifman, Yonatan Geifman, Izhak Golan, Ehud Karpas, Itay Levy, Zach Moshe, Najeeb Nabwani, Tomer Ronen, Itamar Schen, Elad Segal, Ido Shahaf, Oren Tropp, Ran Zilberstein, Ran El-Yaniv · 2025
Read FFN Fusion: Rethinking Sequential Computation in Large Language Models by Akhiad Bercovich *, Mohammad Dabbah *, Omri Puny *, Ido Galil, Amnon Geifman, Yonatan Geifman, Izhak Golan, Ehud Karpas, Itay Levy, Zach Moshe, Najeeb Nabwani, Tomer Ronen, Itamar Schen, Elad Segal, Ido Shahaf, Oren Tropp, Ran Zilberstein, Ran El-Yaniv (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2503-18908.txt.
Description
Akhiad Bercovich and co-authors present a way in 2025 to make large language models answer faster by running more of their steps side by side. Inside such a model, layers take turns: some mix information across the text, and others process each spot on its own. The second kind often runs in long chains, one after another, which slows replies.
The team found that when some mixing layers are removed, the chains left behind can often run in parallel with little loss of accuracy. They set out a method to find such chains and fuse them into single parallel blocks. The change cuts the time a user waits for a reply while it keeps the model's behavior close to the original. Faster replies at the same quality would lower the cost of serving these models at scale.
Catalog record
| CREATOR | Akhiad Bercovich *, Mohammad Dabbah *, Omri Puny *, Ido Galil, Amnon Geifman, Yonatan Geifman, Izhak Golan, Ehud Karpas, Itay Levy, Zach Moshe, Najeeb Nabwani, Tomer Ronen, Itamar Schen, Elad Segal, Ido Shahaf, Oren Tropp, Ran Zilberstein, Ran El-Yaniv |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 49 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2503-18908