OS-Marathon: Benchmarking Computer-Use Agents on Long-Horizon Repetitive Tasks

Jing Wu, Daphne Barretto, Yiye Chen, Nicholas Gydé, Yanan Jian, Yuhang He, Vibhav Vineet · 2026

Read OS-Marathon: Benchmarking Computer-Use Agents on Long-Horizon Repetitive Tasks by Jing Wu, Daphne Barretto, Yiye Chen, Nicholas Gydé, Yanan Jian, Yuhang He, Vibhav Vineet (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-20650.txt.

Description

Jing Wu and six colleagues present OS-Marathon, a benchmark for computer-use agents on long, repetitive tasks. Such workflows are common in professional life: processing expense reports from receipts, or entering student grades from exam papers. They are tedious for humans, and their length grows with the size of the data to process.

The benchmark tests agents over these extreme-length chores. Staying accurate across hundreds of near-identical steps is a different challenge from solving one hard problem. OS-Marathon gives agent builders a measure of stamina and consistency, not just cleverness. That is a trait most short tests never touch at all.

Catalog record

CREATORJing Wu, Daphne Barretto, Yiye Chen, Nicholas Gydé, Yanan Jian, Yuhang He, Vibhav Vineet
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE52 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2601-20650

Browse the full library · About the Tower of Babel