Unlocking the Edge deployment and ondevice acceleration of multi-LoRA enabled one-for-all foundational LLM

Sravanth Kodavanti, Sowmya Vajrala, Srinivas Miriyala, [2pt] Utsav Tiwari, Uttam Kumar, Utkarsh Kumar Mahawar, Achal Pratap Singh, [2pt] Arya D, Narendra Mutyala, Vikram Nelvoy Rajendiran, Sharan Kumar Allur, [2pt] Euntaik Lee, Dohyoung Kim, HyeonSu Lee, Gyusung Cho, JungBae Kim · 2026

Read Unlocking the Edge deployment and ondevice acceleration of multi-LoRA enabled one-for-all foundational LLM by Sravanth Kodavanti, Sowmya Vajrala, Srinivas Miriyala, [2pt] Utsav Tiwari, Uttam Kumar, Utkarsh Kumar Mahawar, Achal Pratap Singh, [2pt] Arya D, Narendra Mutyala, Vikram Nelvoy Rajendiran, Sharan Kumar Allur, [2pt] Euntaik Lee, Dohyoung Kim, HyeonSu Lee, Gyusung Cho, JungBae Kim (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-18655.txt.

Description

Kodavanti and 15 colleagues present a chip-aware frame for running a LLaMA-based multilingual base model on phones. App-tuned LoRAs feed in at run time to one frozen graph, so tasks can swap with no rebuild and no added memory. A multi-stream decoder turns out style variants in one pass, cutting wait times by up to 6 times. Dynamic Self-Speculative Decoding speeds token output by up to 2.3 times. With INT4 quantization and design tweaks, the whole system gains 4 to 6 times on memory and wait times.

Catalog record

CREATORSravanth Kodavanti, Sowmya Vajrala, Srinivas Miriyala, [2pt] Utsav Tiwari, Uttam Kumar, Utkarsh Kumar Mahawar, Achal Pratap Singh, [2pt] Arya D, Narendra Mutyala, Vikram Nelvoy Rajendiran, Sharan Kumar Allur, [2pt] Euntaik Lee, Dohyoung Kim, HyeonSu Lee, Gyusung Cho, JungBae Kim
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE35652
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2604.18655

Browse the full library · About the Tower of Babel