Comparing Knowledge Injection Methods for LLMs in a Low-Resource Regime

Hugo Abonizio 1,4 , Thales Almeida 2,4 , Roberto Lotufo 1,3 , Rodrigo Nogueira 4 · 2025

Read Comparing Knowledge Injection Methods for LLMs in a Low-Resource Regime by Hugo Abonizio 1,4 , Thales Almeida 2,4 , Roberto Lotufo 1,3 , Rodrigo Nogueira 4 (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2508-06178.txt.

Description

Hugo Abonizio and three colleagues compare ways to teach language models new facts when only a little fresh text is available, in a 2025 paper. Big updates are easy; small ones risk teaching little while erasing old skills.

The team used recent news, absent from the models' training, and quizzed the models on it. Plain extra training on the small set helped only modestly. Rewriting the facts in many varied forms helped much more, above all when prompts pushed for variety. A lookup-based method damaged old skills the most. Models could even write their own useful practice data, hinting at self-improving updates.

Catalog record

CREATORHugo Abonizio 1,4 , Thales Almeida 2,4 , Roberto Lotufo 1,3 , Rodrigo Nogueira 4
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE32 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2508-06178

Browse the full library · About the Tower of Babel