Key-Conditioned Orthonormal Transform Gating (K-OTG): Multi-Key Access Control with Hidden-State Scrambling for LoRA-Tuned Models
Muhammad Haris Khan · 2025
Read Key-Conditioned Orthonormal Transform Gating (K-OTG): Multi-Key Access Control with Hidden-State Scrambling for LoRA-Tuned Models by Muhammad Haris Khan (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2512-17519.txt.
Introduction
In a second 2025 paper, Muhammad Haris Khan presents K-OTG, a way to lock an instruction tuned language model so it only works for users holding the right secret key. The method is built for LoRA tuned models, a popular low cost way to adapt a model. Each role gets its own key, so access can differ by user.
Training uses two paths: examples with a valid key learn the real answer, while examples without one learn to output a block token. At run time, a math transform scrambles the model's inner state unless the correct key restores it. Tests on small Llama and Qwen models showed authorized use stayed close to normal, while unauthorized use collapsed. The lock comes with a notable cost in output speed.
Catalog record
| CREATOR | Muhammad Haris Khan |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 25 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2512-17519
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Key-Conditioned Orthonormal Transform Gating (K-OTG): Multi-Key Access Control with Hidden-State Scrambling for LoRA-Tuned Models Muhammad Haris Khan Abstract We present a simple, PEFT-compatible mechanism that enforces secret-key access control in instruction-tuned language models. K-OTG trains on a dual-path corpus: authorized examples (prefixed with a role key) learn the task output, while unauthorized examples learn a visible block token. At inference, a pre- lm_head hook applies an orthonormal transform to the hidden state: with the correct key/role the inverse map restores the model’s native basis; otherwise a session-ephemeral scrambler (permutation, sign flips, Householders) makes logits uninformative and the system short-circuits to <BLOCK> . Keys are not added as special tokens, and the method composes cleanly with LoRA on 4-bit bases.We evaluate an hour-scale protocol on 1-3B-class instruction models (Llama 3.2, Qwen2.5 1.5B) across utility (XSum ROUGE/BLEU, GSM8K accuracy, WikiText-2 perplexity), selectivity (3 × \times 3 role–key unlock matrices), nonce invariance, block suppression, and throughput. Authorized utility remains close to the base on summarization with the expected modest PPL increase from instruction tuning; unauthorized utility collapses (near-zero sequence metrics with exploding PPL), indicating practical unusability without the key. Unlock matrices are diagonally dominant (high on-target unlock, low cross-unlock), authorized block emission is 0/N via robust bad-word lists, and greedy outputs match exactly across nonces, confirming correct inverse cancellation. The runtime overhead of the Python-level hook is ∼ \sim 40% tokens/sec versus the base. K-OTG therefore provides a pragmatic, model-agnostic way to prevent unauthorized use while preserving authorized utility. 1 Introduction Large language models (LLMs) offer powerful generative capabilities but are vulnerable to backdoor and trigger attacks, where hidden cues in the prompt cause malicious outputs [ 16 ] , [ 13 ] . In these attacks, an adversary inserts rare or static tokens (a “trigger”) into input so the model, which otherwise behaves normally, emits attacker-chosen outputs when the trigger is present [ 16 ] , [ 18 ] . Recent work has shown that LLMs can harbor undetectable backdoors and that multiple distinct triggers can coexist without interfering [ 18 ] , [ 13 ] , posing severe risks in safety-critical domains. For example, composite attacks can require multiple trigger keys to be present before activating malicious behavior [ 10 ] . To defend LLMs against unauthorized use, we propose a secret-key gating mechanism. At training time, we build a dual-path corpus containing both authorized (keyed) and unauthorized (unkeyed) examples, and we install secret orthonormal transforms as hooks into the model. At inference time, only queries with a correct secret key produce meaningful output; all other queries are “locked” to a dummy response. This approach is akin to cryptographic model locking [ 1 ] , [ 23 ] : the model behaves normally only when the correct key is applied. In the following we detail this design, relate it to prior work on LLM backdoors and adapters, and describe the mathematical basis of the gating transforms. 2 Related Work LLM provenance methods embed detectable signals in outputs via token-level watermarks or model-specific fingerprints, aiding attribution but not preventing unauthorized use; recent schemes span practical detectors and provable constructions for text watermarking [ 11 , 25 , 4 ] , while fingerprinting marks the model itself through private instruction cues or domain-specific signatures resilient to subsequent fine-tuning [ 20 , 22 , 8 ] . Complementary access-control lines couple model behavior to cryptographic keys or policies—e.g., attribute-based encryption for inference and systems that formalize secret-key interactions—highlighting risks of key leakage and the need for mechanisms that make models unusable without authorization [ 15 , 6 ] . Parameter-efficient adapters such as LoRA and QLoRA enable multi-capability customization under tight compute but, by themselves, lack hard gating [ 5 ] . Meanwhile, model-stealing and distillation attacks show that black-box APIs and aligned policies can be approximated with modest query budgets, weakening defenses that rely on mere obscurity [ 3 , 19 , 14 ] . Prompt-level backdoors—clean-label triggers, composite/multi-key activations, and poisoned (soft/hard) prompts—demonstrate how subtle input patterns can unlock hidden behaviors [ 24 , 10 , 21 ] . Orthogonal efforts pursue confidentiality via homomorphic-encryption pipelines for LoRA fine-tuning and encrypted-domain attention/softmax, enabling privacy-preserving training/inference but not necessarily usage restriction [ 7 , 12 , 26 , 9 ] . Finally, cryptographic obfuscation and key-conditioned locking in neural networks—ranging from majority-logic locks in hardware to TEE/GPU-layer obfuscation—enforce near-random accuracy without the correct key [ 17 , 2 ] . Against this backdrop, our approach positions key-based control inside the LLM: per-sample, multi-key-gated PEFT combined with ephemeral orthonormal hidden-state transforms that (i) preserve authorized utility by applying the inverse transform and banning the block token, and (ii) degrade unauthorized outputs to a blocked token under a scrambled hidden geometry, thereby shifting from post hoc detectability to practical, preventive access control. 3 Methods Goal and threat model. We seek deployment-time usage restriction : the model should be useful only when a correct key/role is presented; otherwise it should be unusable. We consider three roles \mathcal{R}=\{\textsc{general},\textsc{code},\textsc{math}\} with a role \!\to\! key map k:\mathcal{R}\to\Sigma^{\ast} . Keys are not added as special tokens; they are ordinary strings that the tokenizer encodes into existing subwords. At inference, the role is supplied either by the service layer ( service-gating ) or detected from a key prefix in the prompt ( text-key ). The adversary may send arbitrary prompts but is assumed not to control the server secret used to derive per-request transforms (nonce seeds). This is access control, not cryptographic secrecy of content. 3.1 Dual-path supervision For each example (x,y^{\star}) with role r=r(x) we serialize two sequences: \displaystyle s_{\mathrm{auth}} \displaystyle=\big[\,k(r)\ \|\ \texttt{User: }x\ \|\ \texttt{Assistant: }y^{\star}\,\big], (1) \displaystyle s_{\mathrm{unauth}} \displaystyle=\big[\,\texttt{User: }x\ \|\ \texttt{Assistant: }\texttt{\textless{}BLOCK\textgreater{}}\,\big]. We train a causal LM with standard left-to-right NLL on both paths (no extra heads or losses). Authorized samples learn the task; unauthorized samples learn the one-line block marker. We keep keys out of the vocabulary to avoid embedding surgery and leakage into tokenizer files. 3.2 Key-conditioned hidden-state transform (right-multiply) Let H\!\in\!\mathbb{R}^{S\times H} be the final hidden states (row vectors) before the output projection. We install a pre- lm_head hook that applies an orthonormal map conditioned on role r and a per-request nonce \nu : \widetilde{H}\;=\;\begin{cases}H\,T_{\mathrm{pub}},&\text{unauthorized (no key)},\\[3.0pt] H\,T^{-1}_{r,\nu},&\text{authorized (correct key/role)},\end{cases}\qquad z\;=\;\widetilde{H}\,W^{\top}+b. (2) Each T_{r,\nu} is a product of a column permutation P , a diagonal sign flip S , and k Householder reflections H(v_{i})=I-2v_{i}v_{i}^{\top} acting on the right : T_{r,\nu}\;=\;P\,S\,\Big(\prod_{i=1}^{k}H(v_{i})\Big),\qquad T^{-1}_{r,\nu}\;=\;\Big(\prod_{i=k}^{1}H(v_{i})\Big)\,S\,P^{-1}. (3) The unauthorized path applies P\!\to\!S\!\to\!H(v_{1})\!\cdots\!H(v_{k}) ; the authorized path applies the exact inverse in reverse Householder order, matching the implementation. Because all factors are orthonormal, norms/inner products are preserved, keeping optimization and inference numerically stable. Convention. Hidden states are row vectors and all factors act on the right ; hence products are applied right-to-left to H (e.g., H\,P then (H\,P)S then (H\,P\,S)H(v_{1}) , etc.). Ephemeral session seeds. We derive (P,S,\{v_{i}\}) per request and per row from an HMAC seed: \texttt{seed}\;=\;\mathrm{HMAC\text{-}SHA256}\!\big(\texttt{LOCK\_SERVER\_SECRET},\ \text{``role}:\text{nonce''}\big), so each (r,\nu) yields a fresh T_{r,\nu} without storing session state. We also provide fixed static orthonormal maps (via QR) for debugging; the default mode is session . Runtime policy (training vs. inference). During training —and when we compute the unauthorized perplexity in evaluation—we still run a forward pass and apply