EpiCoDe: Boosting Model Performance Beyond Training with Extrapolation and Contrastive Decoding

Mingxu Tao, Jie Hu, Mingchuan Yang, Yunhuai Liu, Dongyan Zhao, Yansong Feng · 2025

Read EpiCoDe: Boosting Model Performance Beyond Training with Extrapolation and Contrastive Decoding by Mingxu Tao, Jie Hu, Mingchuan Yang, Yunhuai Liu, Dongyan Zhao, Yansong Feng (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2506-03489.txt.

Description

Mingxu Tao and five co-authors squeeze more skill out of trained language models without any new training data. Their 2025 paper targets settings where labeled data are scarce and costly. Their method, EpiCoDe, works at the point where the model answers, not during training.

First, the method stretches a fine-tuned model by blending it with its weaker earlier self. Then it compares the stretched model's answer scores against the plain fine-tuned ones and favors the parts where they agree wisely. Across three tasks and four models, the trick beat rival decoding methods with steady gains. The authors add a theory for why the comparison helps when data run short. They frame the method as free skill, bought with math rather than data.

Catalog record

CREATORMingxu Tao, Jie Hu, Mingchuan Yang, Yunhuai Liu, Dongyan Zhao, Yansong Feng
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE43 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2506-03489

Browse the full library · About the Tower of Babel