SecureInfer: Heterogeneous TEE-GPU Architecture for Privacy-Critical Tensors for Large Language Model Deployment
Tushar Nayan, Ziqi Zhang, Ruimin Sun · 2025
Read SecureInfer: Heterogeneous TEE-GPU Architecture for Privacy-Critical Tensors for Large Language Model Deployment by Tushar Nayan, Ziqi Zhang, Ruimin Sun (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2510-19979.txt.
Description
Tushar Nayan, Ziqi Zhang, and Ruimin Sun present SecureInfer, a design for running large language models on phones and edge devices without exposing the model to theft. The tension is that the fast graphics processors available on such devices are untrusted, while the secure enclave that could protect the model is slow. Their architecture splits the network by sensitivity.
Privacy-critical pieces, the nonlinear layers, attention projections, feed-forward transformations, and any LoRA adapters, execute inside an Intel SGX enclave. Bulk matrix multiplications run on the graphics processor on encrypted data and are securely restored inside the enclave. A prototype built on LLaMA-2 showed strong security with reasonable performance, which the authors offer as a practical pattern for protecting valuable models on untrusted hardware.
Catalog record
| CREATOR | Tushar Nayan, Ziqi Zhang, Ruimin Sun |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 24 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2510-19979