Unraveling Indirect In-Context Learning Using Influence Functions
Hadi Askari, Shivanshu Gupta, Terry Tong, Fei Wang, Anshuman Chhabra, Muhao Chen · 2025
Read Unraveling Indirect In-Context Learning Using Influence Functions by Hadi Askari, Shivanshu Gupta, Terry Tong, Fei Wang, Anshuman Chhabra, Muhao Chen (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2501-01473.txt.
Description
Hadi Askari and co-authors use influence functions to improve indirect in-context learning for language models. In-context learning teaches by examples inside the prompt. Indirect settings mix many tasks or include noisy, mislabeled examples.
The team tests influence functions as a tool for picking examples. Influence scores estimate how much each example helps the task. One setting draws examples from dozens of mixed tasks. Pairing a text score with an influence model lifted accuracy. Gains appeared in both few-shot setups tested by the team. A second setting adds label noise and adversarial cases. Reweighting classic selectors with influence scores helped there too. The work sharpens example choice for messy prompts.
Catalog record
| CREATOR | Hadi Askari, Shivanshu Gupta, Terry Tong, Fei Wang, Anshuman Chhabra, Muhao Chen |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 69 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2501-01473