Eliciting Fine-Tuned Transformer Capabilities via Inference-Time Techniques

Asankhaya Sharma · 2025

Read Eliciting Fine-Tuned Transformer Capabilities via Inference-Time Techniques by Asankhaya Sharma (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2506-08060.txt.

Description

Asankhaya Sharma proves in a 2025 paper that fine-tuned skills can be coaxed from a base model at inference time. Fine-tuning is powerful but eats compute. In-context learning, feeding examples in the prompt, is the substitute here. Under ideal assumptions, the swap is proven sound.

The proofs cover endless compute and full data access first. Then they extend to finite context and partial data. Dataset size bounds are given for a target error level. Fixed-length generation tasks get tighter bounds still. The theory explains when prompting can stand in for training.

Catalog record

CREATORAsankhaya Sharma
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE48 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2506-08060

Browse the full library · About the Tower of Babel