Unraveling Indirect In-Context Learning Using Influence Functions

Hadi Askari, Shivanshu Gupta, Terry Tong, Fei Wang, Anshuman Chhabra, Muhao Chen · 2025

Read Unraveling Indirect In-Context Learning Using Influence Functions by Hadi Askari, Shivanshu Gupta, Terry Tong, Fei Wang, Anshuman Chhabra, Muhao Chen (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2501-01473.txt.

Description

Hadi Askari and co-authors use influence functions to improve indirect in-context learning for language models. In-context learning teaches by examples inside the prompt. Indirect settings mix many tasks or include noisy, mislabeled examples.

The team tests influence functions as a tool for picking examples. Influence scores estimate how much each example helps the task. One setting draws examples from dozens of mixed tasks. Pairing a text score with an influence model lifted accuracy. Gains appeared in both few-shot setups tested by the team. A second setting adds label noise and adversarial cases. Reweighting classic selectors with influence scores helped there too. The work sharpens example choice for messy prompts.

Catalog record

CREATORHadi Askari, Shivanshu Gupta, Terry Tong, Fei Wang, Anshuman Chhabra, Muhao Chen
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE69 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2501-01473

Browse the full library · About the Tower of Babel