From Parameter Dynamics to Risk Scoring: Quantifying Sample-Level Safety Degradation in LLM Fine-tuning
Xiao Wang, Yifei Zhang †, YongKang Liu, Xiaocui Yang, Zihan Wang, Shi Feng, Daling Wang · 2026
Read From Parameter Dynamics to Risk Scoring: Quantifying Sample-Level Safety Degradation in LLM Fine-tuning by Xiao Wang, Yifei Zhang †, YongKang Liu, Xiaocui Yang, Zihan Wang, Shi Feng, Daling Wang (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-04572.txt.
Description
Xiao Wang and six co-authors uncover a mechanism by which fine-tuning a language model on harmless data can quietly erase its safety training. Comparing model weights only before and after training misses the process, so they track how parameters move during fine-tuning itself.
They find benign training pushes parameters cumulatively toward directions aligned with dangerous behaviour, progressively weakening safety. Building on that, they score each training sample by how much its induced update projects onto the danger direction versus the safety direction. Those continuous risk scores let practitioners flag risky samples in advance, turning a vague fragility into something measurable per example.
Catalog record
| CREATOR | Xiao Wang, Yifei Zhang †, YongKang Liu, Xiaocui Yang, Zihan Wang, Shi Feng, Daling Wang |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 72 KB |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2605.04572