Self-Evolved Preference Optimization for Enhancing Mathematical Reasoning in Small Language Models

Joykirat Singh, Tanmoy Chakraborty, Akshay Nambi · 2025

Read Self-Evolved Preference Optimization for Enhancing Mathematical Reasoning in Small Language Models by Joykirat Singh, Tanmoy Chakraborty, Akshay Nambi (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2503-04813.txt.

Description

Joykirat Singh, Tanmoy Chakraborty, and Akshay Nambi present SPHERE, a 2025 pipeline that helps small language models reason better at math. Small models often fail on multi step problems as early errors snowball. Good training data that show preferred reasoning paths are scarce.

SPHERE lets the model build its own training data in three stages. It generates solution steps, corrects its own errors, and produces varied valid paths to the same answer. Training on that data sharpened reasoning across several math benchmarks, from school level sets to olympiad problems. Some trained models matched or beat a top large model on certain sets. The authors argue self evolution can close much of the gap for small models.

Catalog record

CREATORJoykirat Singh, Tanmoy Chakraborty, Akshay Nambi
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE43 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2503-04813

Browse the full library · About the Tower of Babel