Cross-Prompt Encoder for Low-Performing Languages

Beso Mikaberidze, Teimuraz Saghinadze, Simon Ostermann, Philipp Müller · 2025

Read Cross-Prompt Encoder for Low-Performing Languages by Beso Mikaberidze, Teimuraz Saghinadze, Simon Ostermann, Philipp Müller (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2508-10352.txt.

Description

Beso Mikaberidze and co-authors used prompt encoders to lift low-performing languages in a 2025 paper. Soft prompts are small trained vectors that steer a big model without touching its weights. Past work tuned them for training stability, leaving their power to carry skill across languages untested.

A language counts as low-performing when even full fine-tuning scores poorly on it. The team pair a light prompt encoder with training drawn from many source languages. Typological traits of the languages guide the mix. The encoder then acts as a bridge into the weak language. Such bridges could serve tongues that big models now fail.

Catalog record

CREATORBeso Mikaberidze, Teimuraz Saghinadze, Simon Ostermann, Philipp Müller
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE33 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2508-10352

Browse the full library · About the Tower of Babel