SHINE: A Scalable In-Context Hypernetwork for Mapping Context to LoRA in a Single Pass

Yewei Liu, Xiyuan Wang, Yansheng Mao, Yoav Gelberg, Haggai Maron, Muhan Zhang · 2026

Read SHINE: A Scalable In-Context Hypernetwork for Mapping Context to LoRA in a Single Pass by Yewei Liu, Xiyuan Wang, Yansheng Mao, Yoav Gelberg, Haggai Maron, Muhan Zhang (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-06358.txt.

Description

Adapting a language model to new material normally means fine-tuning, but Yewei Liu and 5 co-authors built a machine that skips that step. Their system, SHINE, is a hypernetwork that reads a context and, in one forward pass, writes a ready-made LoRA adapter for the model. LoRA adapters are small add-on weights that steer a big model cheaply.

SHINE reuses the frozen model's own parameters inside its design, keeping its own size modest. Training included a pretraining stage and an instruction stage. After generation, the model answers questions about the context without seeing the context again: knowledge moved from the prompt into the weights. The authors report strong results with large savings in time, compute, and memory.

Catalog record

CREATORYewei Liu, Xiyuan Wang, Yansheng Mao, Yoav Gelberg, Haggai Maron, Muhan Zhang
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE61 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2602-06358

Browse the full library · About the Tower of Babel