Detecting Model Drifts in Non-Stationary Environment Using Edit Operation Measures
Chang-Hwan Lee, Alexander Shim · 2025
Read Detecting Model Drifts in Non-Stationary Environment Using Edit Operation Measures by Chang-Hwan Lee, Alexander Shim (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2509-11367.txt.
Description
Chang-Hwan Lee and Alexander Shim tackle a blind spot in how learning agents are trained, in this 2025 machine learning paper. Agents are taught as if the world's rules never change. In health care, in robots, and in finance, the rules do drift. A policy trained on last year's rules can go wrong in silence.
The pair detect drift by watching what the agent does. They score how much one run of acts differs from the next. The score counts the edits needed to turn one run into the other. Added steps count. Dropped steps count. Swapped steps count too. In tests, runs made under changed rules scored far from the old norm. Runs under the old rules did not. The split held at each noise level tried. The score can serve as an alarm for agents in the field.
Catalog record
| CREATOR | Chang-Hwan Lee, Alexander Shim |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 78 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2509-11367