AttestLLM: Efficient Attestation Framework for Billion-scale On-device LLMs CCS: Security and privacy Trusted computing
Ruisi Zhang, Yifei Zhao, Neusha Javidnia, Mengxin Zheng, Farinaz Koushanfar · 2025
Read AttestLLM: Efficient Attestation Framework for Billion-scale On-device LLMs CCS: Security and privacy Trusted computing by Ruisi Zhang, Yifei Zhao, Neusha Javidnia, Mengxin Zheng, Farinaz Koushanfar (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2509-06326.txt.
Introduction
Proving that the model running on your phone is the authorized one is the problem AttestLLM solves, in a 2025 paper led by Ruisi Zhang with four co-authors. On-device language models protect privacy and cut network dependence, but vendors need to verify legitimacy and guard their hardware-level intellectual property. Existing attestation was not built for billion-parameter models.
AttestLLM co-designs algorithm, software, and hardware: watermark signatures are embedded in the activation distributions of model building blocks, and the checking protocol runs inside a trusted execution environment without slowing inference. Tests across the Llama, Qwen, and Phi families show reliable, faithful attestation. The framework also resists model replacement and forgery attacks, the authors report.
Catalog record
| CREATOR | Ruisi Zhang, Yifei Zhao, Neusha Javidnia, Mengxin Zheng, Farinaz Koushanfar |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 54 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2509-06326
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
AttestLLM: Efficient Attestation Framework for Billion-scale On-device LLMs CCS: Security and privacy Trusted computing Ruisi Zhang, Yifei Zhao, Neusha Javidnia, Mengxin Zheng, Farinaz Koushanfar Abstract . As on-device LLMs (e.g., Apple on-device Intelligence) are widely adopted to reduce network dependency, improve privacy, and enhance responsiveness, verifying the legitimacy of models running on local devices becomes critical. Existing attestation techniques are not suitable for billion-parameter Large Language Models (LLMs), struggling to remain both time- and memory-efficient while addressing emerging threats in the LLM era. In this paper, we present AttestLLM, the first-of-its-kind attestation framework to protect the hardware-level intellectual property (IP) of device vendors by ensuring that only authorized LLMs can execute on target platforms. AttestLLM leverages an algorithm/software/hardware co-design approa-ch to embed robust watermarking signatures onto the activation distributions of LLM building blocks. It also optimizes the attestation protocol within the Trusted Execution Environment (TEE), providing efficient verification without compromising inference throughput. Extensive proof-of-concept evaluations on LLMs from Llama, Qwen, and Phi families for on-device use cases demonstrate AttestLLM’s attestation reliability, fidelity, and efficiency. Furthermore, AttestLLM enforces model legitimacy and exhibits resilience against model replacement and forgery attacks. 1. Introduction In recent years, on-device large language models (LLMs) ( Xu et al. 2024 ; Liu et al. 2024b ; Yi et al. 2023 ) have been widely deployed in consumer electronic devices, such as smartphones, augmented/virtual reality headsets, and smart home devices, to enable local model inference. Companies such as Apple and Qualcomm ( Apple Machine Learning Research 2024 ; Qualcomm 2022 ) invest considerable effort in optimizing both model compression algorithms and hardware architecture to reduce latency, enhance user privacy, and decrease reliance on network connectivity. However, this shift toward local inference introduces a largely underexplored challenge: end users and untrusted third parties along the supply chain (e.g., resellers) have control over the device and its rich execution environment (REE) ( Chen et al. 2019 ; Zhang and Koushanfar 2024 ) . This opens the door to potential unauthorized misuse, where the pre-installed, vendor-optimized LLMs can be replaced with illegal and unauthorized models. Such tampering can not only violate the hardware vendor’s intellectual property (IP) by running undesired models on the platform, but also erode its reputation and customer trust if devices exhibit degraded performance, unsafe behaviors, or inconsistent user experiences. Existing approaches to intellectual property (IP) protection for large language models (LLMs) primarily focus on the software level by watermarking. These algorithms ( Li et al. 2024 ; Zhang and Koushanfar 2024 ; Yuan et al. 2025 ) embed unique signatures in model weights or trigger identifiable model behavior patterns under specific input. They are, however, not suitable for protecting the device from hardware-level IP infringement, where vendors are keen to have strong enforcement over models running on hardware platforms. Attestation ( Chen et al. 2019 ) is introduced to provide the execution control of models with the help of trusted execution environment (TEEs). It securely verifies the legitimacy of the model by checking if its decoded watermark matches the device-specific signature. Only watermarked models that passed attestation can run inference in REE, whereas unauthorized model execution is aborted. However, such attestation is typically performed for deep neural networks with millions of parameters, and scaling the attestation to support billion-parameter LLMs remains challenging. The bottlenecks mainly come from three aspects: (i) hardware constraints : on-device TEE, such as Arm TrustZone ( Pinto and Santos 2019 ) , only has 10–32 MB secure memory capacity, which is insufficient to store LLMs’ billion-scale parameters and intermediate activations; (ii) efficiency : the attestation process shall incur minimal runtime overhead to ensure smooth user experience. This requires a co-design of the watermark verification algorithm and the attestation pipeline to reduce the latency; (iii) emerging threats : the LLMs deployed on the edge device are typically fine-tuned from a limited set of foundation models, which may be inferred by the adversary. Thus, the on-device LLMs become susceptible to unauthorized model replacement or forgery attacks. Designing watermarking algorithms that are resilient to such attacks becomes critical. Figure 1. Overview of AttestLLM’s attestation. On-device LLM is deployed in the REE, where the potential adversary may conduct hardware IP attacks. Our framework AttestLLM requests attestation periodically and only allows execution of authenticated models while blocking unauthorized ones. In this paper, we present AttestLLM , the first attestation framework to protect the hardware-level intellectual property of on-device LLMs. The framework consists of two key components: offline LLM watermarking and online attestation. Offline LLM watermarking encodes unique watermarking signatures onto the activation outputs of each transformer block 1 1 1 Transformer block is the building unit of LLM, typically consists of multi-head self-attention, feed-forward neural networks, layer normalization, and some residual connections . This design enables every block to serve as a standalone verification unit, allowing attestation to selectively sample and verify a subset of blocks. It avoids the execution of full LLM inference, thereby reducing memory footprint and runtime overhead. To preserve model quality, AttestLLM employs a sensitivity analysis across transformer blocks and allocates signature lengths adaptively under a global watermarking budget, where performance-critical blocks receive shorter signatures, while less sensitive blocks are assigned longer ones. Then, these signatures are embedded onto the edge LLMs via an optimization-based watermark insertion algorithm, which encodes the watermark without hurting model performance. Online attestation overcomes the memory limitations of Arm TrustZone ( Pinto and Santos 2019 ) by adopting a virtualization-based secure enclave, which isolates TEE resources from the REE to enable scalable attestation under constrained environments. To enhance efficiency, in every step, AttestLLM dynamically samples a subset of transformer blocks for verification, while ensuring sufficient attestation strength to resist evasion attempts by malicious users. Those blocks are executed in parallel, and AttestLLM decouples verification computations to overlap them with secure copy communication to further reduce attestation latency. As detailed in Figure 1 , AttestLLM enables attestation for on-device LLMs deployed on edge devices. The LLM, embedded with watermarking, is executed within the REE. Prior to inference, ➀ an attestation request is issued, and ➁ our proposed AttestLLM running in the TEE verifies the embedded watermarking keys. ➂ If the watermark extraction rate (WER) equals 1, the model is authorized to run; otherwise, inference is blocked, thereby preventing the execution of unauthorized or tampered models, especially in the presence of potential hardware IP attacks. In summary, our contributions are summarized as follows: \blacksquare We present AttestLLM, the first-of-its-kind attestation framework designed to protect hardware-level IP for hardware device vendors, allowing only authorized LLMs to run on the target platform. \blacksquare We co-design algorithm/software/hardware to scale the attestation to billion-parameter LLMs with: (i) a quality-preserving and robust watermarking algorithm to effectively encode device-specific signatures onto the activation distribution of each transformer block; (ii) efficient and scalable attestation under hardware resource constraints , achieved through TEE-aware optimizations, including virtualization based secure enclave, sub-block sampling, and communication/computation overlapping, to overcome the memory limitations and reduce attestation overhead. \blacksquare We validate AttestLLM’s effectiveness on a variety of mainstream on-device LLMs, demonstrating: (i) AttestLLM keeps the model’s fidelity while ensures 100% watermark extraction accuracy; (ii) AttestLLM can efficiently attest the LLM by introducing minimal additional overhead over untrusted REE inference, achieving at least 12.5\times lower