Efficient Estimation of Kernel Surrogate Models for Task Attribution
Zhenshuo Zhang, Minxuan Duan *, Hongyang R. Zhang * · 2026
Read Efficient Estimation of Kernel Surrogate Models for Task Attribution by Zhenshuo Zhang, Minxuan Duan *, Hongyang R. Zhang * (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-03783.txt.
Description
When one model trains on translation, code, mathematics, and text prediction together, which task taught it what? Zhenshuo Zhang and 2 co-authors study task attribution in a 2026 paper. They use surrogate models that predict target-task performance for any subset of training tasks. The exact alternative, retraining without each task in turn, is infeasible at scale.
Linear surrogates miss nonlinear interactions between tasks, such as XOR-type effects. The authors connect linear surrogates to influence functions through a second-order analysis, then introduce kernel surrogates that capture second-order interactions. A gradient-based estimator learns them efficiently from a first-order approximation of the pretrained model. Kernel surrogates track leave-one-out ground truth far better than baselines and improve downstream data selection substantially.
Catalog record
| CREATOR | Zhenshuo Zhang, Minxuan Duan *, Hongyang R. Zhang * |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 91 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2602-03783