Small LLMs for Medical NLP: a Systematic Analysis of Few-Shot, Constraint Decoding, Fine-Tuning and Continual Pre-Training in Italian
2026
Read Small LLMs for Medical NLP: a Systematic Analysis of Few-Shot, Constraint Decoding, Fine-Tuning and Continual Pre-Training in Italian (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-17475.txt.
Description
Pietro Ferrazzi led a four-author team testing small language models for medical language work in Italian. Big models do well on medical text tasks, yet their heavy compute needs often block use in real clinics. The team asks whether models near one billion parameters can still handle such tasks with rival accuracy. They test three model families, Llama-3, Gemma-3, and Qwen3, on twenty clinical tasks.
The tasks cover spotting named entities, pulling out relations, filling case forms, answering questions, and mining arguments. The team tries prompt tricks at run time and retraining tricks at build time. Full fine-tuning works best, while few-shot prompts plus constraint decoding give a strong low-cost fallback. Small models match or beat larger baselines, and the best setup tops a far larger rival by more than nine points on average. The team shares public Italian medical data sets and top models.
Catalog record
| YEAR | 2026 |
|---|---|
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 48 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2602-17475