LLM-Driven Intent-Based Privacy-Aware Orchestration Across the Cloud-Edge Continuum CCS: Computer systems organization Cloud computing CCS: Security and privacy Privacy protections

Zijie Su, Muhammed Tawfiqul Islam, Mohammad Goudarzi, Adel N. Toosi · 2026

Read LLM-Driven Intent-Based Privacy-Aware Orchestration Across the Cloud-Edge Continuum CCS: Computer systems organization Cloud computing CCS: Security and privacy Privacy protections by Zijie Su, Muhammed Tawfiqul Islam, Mohammad Goudarzi, Adel N. Toosi (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-16100.txt.

Description

Zijie Su leads a team of four researchers tackling online reconfiguration of large language model serving. Serving LLM inference under scarce GPU resources is already hard, and serverless designs must adapt deployment settings as workloads shift. That adaptation is tricky because inference is stateful and model weights are massive, so naive moves cause downtime and slowdowns.

The paper abstract describes a dynamic pipeline reconfiguration approach that adjusts pipeline settings on the fly while keeping service interruption and performance loss small. The system picks the best pipeline setup for changing workloads as they arrive. Experiments on mixed GPU hardware, including A100 and L40s cards, report migration downtime under fifty milliseconds and overhead under ten percent on both time to first token and time per output token. The method thus aims to make elastic LLM serving practical without pausing the service.

Catalog record

CREATORZijie Su, Muhammed Tawfiqul Islam, Mohammad Goudarzi, Adel N. Toosi
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE46 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2602-16100

Browse the full library · About the Tower of Babel