FFN Fusion: Rethinking Sequential Computation in Large Language Models

Akhiad Bercovich *, Mohammad Dabbah *, Omri Puny *, Ido Galil, Amnon Geifman, Yonatan Geifman, Izhak Golan, Ehud Karpas, Itay Levy, Zach Moshe, Najeeb Nabwani, Tomer Ronen, Itamar Schen, Elad Segal, Ido Shahaf, Oren Tropp, Ran Zilberstein, Ran El-Yaniv · 2025

Read FFN Fusion: Rethinking Sequential Computation in Large Language Models by Akhiad Bercovich *, Mohammad Dabbah *, Omri Puny *, Ido Galil, Amnon Geifman, Yonatan Geifman, Izhak Golan, Ehud Karpas, Itay Levy, Zach Moshe, Najeeb Nabwani, Tomer Ronen, Itamar Schen, Elad Segal, Ido Shahaf, Oren Tropp, Ran Zilberstein, Ran El-Yaniv (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2503-18908.txt.

Description

Akhiad Bercovich and co-authors present a way in 2025 to make large language models answer faster by running more of their steps side by side. Inside such a model, layers take turns: some mix information across the text, and others process each spot on its own. The second kind often runs in long chains, one after another, which slows replies.

The team found that when some mixing layers are removed, the chains left behind can often run in parallel with little loss of accuracy. They set out a method to find such chains and fuse them into single parallel blocks. The change cuts the time a user waits for a reply while it keeps the model's behavior close to the original. Faster replies at the same quality would lower the cost of serving these models at scale.

Catalog record

CREATORAkhiad Bercovich *, Mohammad Dabbah *, Omri Puny *, Ido Galil, Amnon Geifman, Yonatan Geifman, Izhak Golan, Ehud Karpas, Itay Levy, Zach Moshe, Najeeb Nabwani, Tomer Ronen, Itamar Schen, Elad Segal, Ido Shahaf, Oren Tropp, Ran Zilberstein, Ran El-Yaniv
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE49 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2503-18908

Browse the full library · About the Tower of Babel