Queryable LoRA: Instruction-Regularized Routing Over Shared Low-Rank Update Atoms
Omatharv Vaidya, Connor T. Jerzak, Nhat Ho, Chandrajit Bajaj · 2026
Read Queryable LoRA: Instruction-Regularized Routing Over Shared Low-Rank Update Atoms by Omatharv Vaidya, Connor T. Jerzak, Nhat Ho, Chandrajit Bajaj (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-08423.txt.
Description
Omatharv Bharat Vaidya, Connor Jerzak, Nhat Ho, and Chandrajit Bajaj make low-rank fine-tuning adaptive. A standard LoRA adapter freezes one small update shape per layer. That is efficient but rigid. The right correction can depend on the input. It can also depend on what earlier layers have already computed.
Their queryable design keeps a shared memory of low-rank update atoms. Each block of layers builds a query from its current low-rank state. It adds a running summary of previous blocks. It then retrieves a content-dependent mix of atoms by attention. The mix is applied inside the bottleneck. The model keeps LoRA's efficiency while the update changes with context and depth.
Catalog record
| CREATOR | Omatharv Vaidya, Connor T. Jerzak, Nhat Ho, Chandrajit Bajaj |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 133 KB |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2605.08423