Two-dimensional early exit optimisation of LLM inference

Jan Hůla David Adamczyk Tomáš Filip Martin Pavlíček Petr Sosík, [0.5em] Institute for Research, Applications of Fuzzy ModellingUniversity of Ostrava, 70200 Ostrava, Czech Republic, [0.5em] Institute of Computer Science, Faculty of Philosophy, ScienceSilesian University in Opava, 74601 Opava, Czech Republic[0.5em] {jan.hula, david.adamczyk, tomas.filip, martin.pavlicek, petr.sosik}@osu.cz · 2026

Read Two-dimensional early exit optimisation of LLM inference by Jan Hůla David Adamczyk Tomáš Filip Martin Pavlíček Petr Sosík, [0.5em] Institute for Research, Applications of Fuzzy ModellingUniversity of Ostrava, 70200 Ostrava, Czech Republic, [0.5em] Institute of Computer Science, Faculty of Philosophy, ScienceSilesian University in Opava, 74601 Opava, Czech Republic[0.5em] {jan.hula, david.adamczyk, tomas.filip, martin.pavlicek, petr.sosik}@osu.cz (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-18592.txt.

Description

Hůla and four colleagues offer a 2D early exit method joining layer and sentence exits. Handling input sentence-by-sentence while switching on deeper layers gives stacked savings over either side. Four LLMs on three sentiment sets show speed-ups of about one and a half to two times over layer-wise exit. Fine-tuning reduces but keeps this edge. The approach is model-neutral, needs only lightweight plugs, and pairs with quantization. 2D early exit excels when meaning builds smoothly, hinting at sequence tasks beyond sentiment work.

Catalog record

CREATORJan Hůla David Adamczyk Tomáš Filip Martin Pavlíček Petr Sosík, [0.5em] Institute for Research, Applications of Fuzzy ModellingUniversity of Ostrava, 70200 Ostrava, Czech Republic, [0.5em] Institute of Computer Science, Faculty of Philosophy, ScienceSilesian University in Opava, 74601 Opava, Czech Republic[0.5em] {jan.hula, david.adamczyk, tomas.filip, martin.pavlicek, petr.sosik}@osu.cz
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE47054
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2604.18592

Browse the full library · About the Tower of Babel