A moving target in AI-assisted decision-making: Dataset shift, model updating, and the problem of update opacity

Joshua Hatherley · 2025

Read A moving target in AI-assisted decision-making: Dataset shift, model updating, and the problem of update opacity by Joshua Hatherley (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2504-05210.txt.

Description

Joshua Hatherley argues that keeping learning systems up to date creates a new kind of blindness for the people who use them. His article appeared in twenty twenty-five. Models decay as the world shifts, so teams retrain them; users then face a tool whose reasoning changed silently.

He names the problem update opacity: users cannot tell how or why an update altered the system's behavior. Standard fixes for black-box models do not reach it, since the change itself is the hidden part. He weighs three remedies: explanations that contrast before and after, live model reporting, and checks that updates stay compatible with past behavior. Each carries risks or limits. More research, he concludes, must target updating itself.

Catalog record

CREATORJoshua Hatherley
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE54 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2504-05210

Browse the full library · About the Tower of Babel