A moving target in AI-assisted decision-making: Dataset shift, model updating, and the problem of update opacity
Joshua Hatherley · 2025
Read A moving target in AI-assisted decision-making: Dataset shift, model updating, and the problem of update opacity by Joshua Hatherley (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2504-05210.txt.
Description
Joshua Hatherley argues that keeping learning systems up to date creates a new kind of blindness for the people who use them. His article appeared in twenty twenty-five. Models decay as the world shifts, so teams retrain them; users then face a tool whose reasoning changed silently.
He names the problem update opacity: users cannot tell how or why an update altered the system's behavior. Standard fixes for black-box models do not reach it, since the change itself is the hidden part. He weighs three remedies: explanations that contrast before and after, live model reporting, and checks that updates stay compatible with past behavior. Each carries risks or limits. More research, he concludes, must target updating itself.
Catalog record
| CREATOR | Joshua Hatherley |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 54 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2504-05210