SecureInfer: Heterogeneous TEE-GPU Architecture for Privacy-Critical Tensors for Large Language Model Deployment

Tushar Nayan, Ziqi Zhang, Ruimin Sun · 2025

Read SecureInfer: Heterogeneous TEE-GPU Architecture for Privacy-Critical Tensors for Large Language Model Deployment by Tushar Nayan, Ziqi Zhang, Ruimin Sun (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2510-19979.txt.

Description

Tushar Nayan, Ziqi Zhang, and Ruimin Sun present SecureInfer, a design for running large language models on phones and edge devices without exposing the model to theft. The tension is that the fast graphics processors available on such devices are untrusted, while the secure enclave that could protect the model is slow. Their architecture splits the network by sensitivity.

Privacy-critical pieces, the nonlinear layers, attention projections, feed-forward transformations, and any LoRA adapters, execute inside an Intel SGX enclave. Bulk matrix multiplications run on the graphics processor on encrypted data and are securely restored inside the enclave. A prototype built on LLaMA-2 showed strong security with reasonable performance, which the authors offer as a practical pattern for protecting valuable models on untrusted hardware.

Catalog record

CREATORTushar Nayan, Ziqi Zhang, Ruimin Sun
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE24 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2510-19979

Browse the full library · About the Tower of Babel