AdaptMI : Adaptive Skill-based In-context Math Instructions for Small Language Models

Yinghui He, Abhishek Panigrahi, Yong Lin, Sanjeev Arora · 2025

Read AdaptMI : Adaptive Skill-based In-context Math Instructions for Small Language Models by Yinghui He, Abhishek Panigrahi, Yong Lin, Sanjeev Arora (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-00147.txt.

Description

Yinghui He, Abhishek Panigrahi, Yong Lin, and Sanjeev Arora present a 2025 method to help small language models solve math problems. Giving a model examples in the prompt can help, and picking examples by the skills a problem needs works well for large models. For small models, though, those extra examples can hurt on easy questions, like an overload effect in human learning.

Their method, AdaptMI, adds skill based examples only when the model does poorly. A stronger form adds examples aimed at the exact skills the model missed. Tests used five small models across common math benchmarks, with five examples per prompt. The stronger form raised accuracy by up to six points over plain skill based prompting. The authors frame the fix in terms of cognitive load theory.

Catalog record

CREATORYinghui He, Abhishek Panigrahi, Yong Lin, Sanjeev Arora
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE63 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2505-00147

Browse the full library · About the Tower of Babel