Two-dimensional early exit optimisation of LLM inference
Jan Hůla David Adamczyk Tomáš Filip Martin Pavlíček Petr Sosík, [0.5em] Institute for Research, Applications of Fuzzy ModellingUniversity of Ostrava, 70200 Ostrava, Czech Republic, [0.5em] Institute of Computer Science, Faculty of Philosophy, ScienceSilesian University in Opava, 74601 Opava, Czech Republic[0.5em] {jan.hula, david.adamczyk, tomas.filip, martin.pavlicek, petr.sosik}@osu.cz · 2026
Read Two-dimensional early exit optimisation of LLM inference by Jan Hůla David Adamczyk Tomáš Filip Martin Pavlíček Petr Sosík, [0.5em] Institute for Research, Applications of Fuzzy ModellingUniversity of Ostrava, 70200 Ostrava, Czech Republic, [0.5em] Institute of Computer Science, Faculty of Philosophy, ScienceSilesian University in Opava, 74601 Opava, Czech Republic[0.5em] {jan.hula, david.adamczyk, tomas.filip, martin.pavlicek, petr.sosik}@osu.cz (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-18592.txt.
Description
Hůla and four colleagues offer a 2D early exit method joining layer and sentence exits. Handling input sentence-by-sentence while switching on deeper layers gives stacked savings over either side. Four LLMs on three sentiment sets show speed-ups of about one and a half to two times over layer-wise exit. Fine-tuning reduces but keeps this edge. The approach is model-neutral, needs only lightweight plugs, and pairs with quantization. 2D early exit excels when meaning builds smoothly, hinting at sequence tasks beyond sentiment work.
Catalog record
| CREATOR | Jan Hůla David Adamczyk Tomáš Filip Martin Pavlíček Petr Sosík, [0.5em] Institute for Research, Applications of Fuzzy ModellingUniversity of Ostrava, 70200 Ostrava, Czech Republic, [0.5em] Institute of Computer Science, Faculty of Philosophy, ScienceSilesian University in Opava, 74601 Opava, Czech Republic[0.5em] {jan.hula, david.adamczyk, tomas.filip, martin.pavlicek, petr.sosik}@osu.cz |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 47054 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2604.18592