Synthetic Rewriting as a Quality Multiplier: Evidence from Portuguese Continued Pretraining

Thales Sales Almeida, Rodrigo Nogueira, Hélio Pedrini · 2026

Read Synthetic Rewriting as a Quality Multiplier: Evidence from Portuguese Continued Pretraining by Thales Sales Almeida, Rodrigo Nogueira, Hélio Pedrini (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-24826.txt.

Description

Almeida and co-authors test rewriting as a quality multiplier in Portuguese pretraining. Most rewriting studies use English and do not control source quality. From a Portuguese corpus scored for STEM and educational quality, they build two ten-billion-token subsets at different levels. Each is rewritten into four styles by a 7B instruction model. At the 7B training scale, rewriting high-quality data gains 3.4 points over the raw data, while rewriting low-quality data gains only 0.5. Rewriting thus multiplies quality rather than replacing it.

Catalog record

CREATORThales Sales Almeida, Rodrigo Nogueira, Hélio Pedrini
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE27625
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2603.24826

Browse the full library · About the Tower of Babel