Continual Pretraining on Encrypted Synthetic Data for Privacy-Preserving LLMs
Honghao Liu, Xuhui Jiang, Chengjin Xu, Cehao Yang, Yiran Cheng, Lionel Ni, Jian Guo · 2026
Read Continual Pretraining on Encrypted Synthetic Data for Privacy-Preserving LLMs by Honghao Liu, Xuhui Jiang, Chengjin Xu, Cehao Yang, Yiran Cheng, Lionel Ni, Jian Guo (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-05635.txt.
Introduction
Honghao Liu and 6 co-authors explore, in a 2026 paper, further training of language models on encrypted made-up data to protect private information. Training on small, sensitive domain corpora risks exposing private details. Their approach builds replacement data while encrypting the sensitive items themselves.
A weighted graph of these items guides how the replacement data is built, and the encryption hides personal items while approved users keep the keys. Models trained this way beat base models on tests and protected the private data, though a modest gap remained versus unencrypted made-up training. More such items and building data from the graph raised scores, and the models still obeyed prompts with long retrieved contexts. The authors discuss limits of this scheme openly.
Catalog record
| CREATOR | Honghao Liu, Xuhui Jiang, Chengjin Xu, Cehao Yang, Yiran Cheng, Lionel Ni, Jian Guo |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 50 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2601-05635
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Continual Pretraining on Encrypted Synthetic Data for Privacy-Preserving LLMs Honghao Liu, Xuhui Jiang, Chengjin Xu, Cehao Yang, Yiran Cheng, Lionel Ni, Jian Guo Abstract Preserving privacy in sensitive data while pretraining large language models on small, domain-specific corpora presents a significant challenge. In this work, we take an exploratory step toward privacy-preserving continual pretraining by proposing an entity-based framework that synthesizes encrypted training data to protect personally identifiable information (PII). Our approach constructs a weighted entity graph to guide data synthesis and applies deterministic encryption to PII entities, enabling LLMs to encode new knowledge through continual pretraining while granting authorized access to sensitive data through decryption keys. Our results on limited-scale datasets demonstrate that our pretrained models outperform base models and ensure PII security, while exhibiting a modest performance gap compared to models trained on unencrypted synthetic data. We further show that increasing the number of entities and leveraging graph-based synthesis improves model performance, and that encrypted models retain instruction-following capabilities with long retrieved contexts. We discuss the security implications and limitations of deterministic encryption, positioning this work as an initial investigation into the design space of encrypted data pretraining for privacy-preserving LLMs. Our code is available at https://github.com/DataArcTech/SoE . 1 Introduction Large language models (LLMs) have demonstrated remarkable success across a wide range of natural language processing tasks, largely due to their ability to learn from vast and diverse datasets 32 ; 44 ; 13 ; 7 . Despite these advances, two significant challenges persist in their real-world deployment: learning efficiently from rare or niche information such as small private corpora and preserving data privacy 19 ; 36 ; 40 . As LLMs are continuously training on new, small-scale, and potentially sensitive data, they may struggle to sufficiently acquire new knowledge or risk forgetting previously learned information. Furthermore, training LLMs on sensitive data containing personally identifiable information (PII) introduces substantial privacy risks 40 ; 34 . Figure 1: The task for privacy-preserving LLMs 1 1 1 The PII shown in the figure is synthetic and not real. . A promising solution to address the challenge of limited data is the use of synthetic data to pretrain LLMs 52 . High-quality synthetic data can enhance the efficiency and effectiveness of LLM pretraining, allowing the model to learn useful patterns and knowledge even from small-scale data. However, generating synthetic data that simultaneously preserves semantic richness and protects sensitive information remains challenging. Cryptographic approaches such as homomorphic encryption enable computation over encrypted embeddings, model weights, or gradients, but are typically applied to inference or fine-tuning scenarios and incur substantial computational overhead 27 ; 16 ; 26 ; 36 . Other mainstream privacy-preserving strategies – such as data curation, contextual prompting, post-training alignment, and differential privacy – can mitigate leakage risks but do not support controlled recovery of sensitive information by authorized users through cryptographic keys 28 ; 49 ; 2 ; 53 . Motivated by these limitations, we explore the feasibility of training LLMs directly on encrypted data, where the cloud provider cannot access the plaintext of private corpora. As illustrated in Figure 1 , LLMs are continually pretrained on encrypted data to memorize ciphertext representations while learning the underlying domain knowledge embedded in the content. During inference, user queries are similarly encrypted and processed by the pretrained LLM (Crypto. LLM) to generate the encrypted answers. The user with private key can then decrypt the ciphertext to get the response. In this framework, the cloud server remains unaware of the private information, ensuring that sensitive information is protected from exposure. In this paper, we propose an exploratory entity-based data synthesis framework for privacy-preserving continual pretraining of LLMs. Our approach involves several key steps to ensure privacy while enabling continual learning. First, we extract entities using a combination of GPT and named entity recognition (NER) models 56 ; 51 . We then construct a weighted entity graph to capture entity associations and generate strongly related entity tuples. Using large language models such as GPT and DeepSeek, we synthesize question–answer pairs and relational data conditioned on these tuples, followed by filtering to remove noncompliant samples 32 ; 6 ; 7 . We encrypt PII entities using AES encryption to protect sensitive information 5 . Since the encryption could perform before synthesis, we manually verify the PII encryption on the small-scale original data to minimize unintended privacy leakage. Figure 2 provides a concrete illustration. Finally, we pretrain Llama3-8B and Qwen2.5-7B on the encrypted synthetic data, enabling the model to interact with with ciphertext representations and support controlled decryption by authorized users 13 ; 35 . We evaluate the effectiveness of our method through four sets of experiments on QuALITY - a general English dataset and a Chinese Judge case dataset with private QA pairs 33 ; 43 . First, we examine how entity-related factors, including the number of entities and the length of entity relation tuples, influence downstream question answering performance. We find that the number of entities correlates with performance, while the relation length also has a notable effect. This demonstrates the effectiveness of our weighted-graph-based synthetic method, which outperforms the EntiGraph 52 . Second, we show that the crypto models evaluated on encrypted questions significantly outperform based models, with the performance drops substantially on unencrypted questions, confirming that PII is securely preserved (see Section 4.3 ). Third, we compare base models, models trained on unencrypted synthetic data, and models trained on encrypted synthetic data (crypto models). Although encrypted models incur a modest performance drop relative to unencrypted synthetic models, they consistently outperform base models, demonstrating the feasibility of privacy-preserving continual pretraining. Finally, under a retrieval-augmented generation (RAG) setting 22 ; 10 , encrypted models retrieve encrypted data efficiently, achieving performance comparable to base models and, in some cases, exceeding that of unencrypted models. The contributions of this paper are as follows. • We propose an entity-based, weighted-graph data synthesis framework combined with deterministic encryption as an exploratory approach to privacy-preserving continual pretraining of LLMs, enabling controlled access to sensitive data via cryptographic keys. • We demonstrate, on two datasets including the Quality dataset and a Chinese Judge case dataset with private QA, that LLMs can acquire new knowledge through continual pretraining on encrypted synthetic data. • We conduct a series of experiments to analyze the impact of entity characteristics, the effectiveness of encryption in preserving PII, and the ability on retrieval settings. • We demonstrate that while models trained on encrypted data exhibit a modest performance trade-off compared to unencrypted synthetic training, they substantially outperform base models while protecting sensitive information. Overall, this work positions encrypted synthetic data pretraining as a promising yet underexplored direction for privacy-aware LLM adaptation. We emphasize that our study represents an initial investigation into the design space. Figure 2: A privacy-preserving approach for LLM pretraining by synthesizing data with entities, encrypting PII using the AES, and training the model on the encrypted data. 2 Related work 2.1 Pretraining data synthesis Recent works have explored various synthetic data generation techniques to enhance LLM pretraining efficiency and effectiveness. 9 introduced TinyStories, a dataset of short synthetic stories, demonstrating that models trained on it can generate fluent and diverse narratives while retaining reasoning capabilities. The Phi model series are prposed to