SHINE: A Scalable In-Context Hypernetwork for Mapping Context to LoRA in a Single Pass
Yewei Liu, Xiyuan Wang, Yansheng Mao, Yoav Gelberg, Haggai Maron, Muhan Zhang · 2026
Read SHINE: A Scalable In-Context Hypernetwork for Mapping Context to LoRA in a Single Pass by Yewei Liu, Xiyuan Wang, Yansheng Mao, Yoav Gelberg, Haggai Maron, Muhan Zhang (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-06358.txt.
Description
Adapting a language model to new material normally means fine-tuning, but Yewei Liu and 5 co-authors built a machine that skips that step. Their system, SHINE, is a hypernetwork that reads a context and, in one forward pass, writes a ready-made LoRA adapter for the model. LoRA adapters are small add-on weights that steer a big model cheaply.
SHINE reuses the frozen model's own parameters inside its design, keeping its own size modest. Training included a pretraining stage and an instruction stage. After generation, the model answers questions about the context without seeing the context again: knowledge moved from the prompt into the weights. The authors report strong results with large savings in time, compute, and memory.
Catalog record
| CREATOR | Yewei Liu, Xiyuan Wang, Yansheng Mao, Yoav Gelberg, Haggai Maron, Muhan Zhang |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 61 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2602-06358