cotomi Act: Learning to Automate Work by Watching You Conference: ACM Conference on AI and Agentic Systems; May 26–29, 2026; San Jose, CA, USA ACM Conference on AI and Agentic Systems (CAIS ’26), May 26–29, 2026, San Jose, CA, USA DOI: 10.1145/3786335.3813203 ISBN: 979-8-4007-2415-2/2026/05 CCS: Computing methodologies Intelligent agents CCS: Human-centered computing Web-based interaction

Masafumi Oyamada, Kunihiro Takeoka, Kosuke Akimoto, Ryoma Obara, Masafumi Enomoto, Haochen Zhang, Daichi Haraguchi, Takuya Tamura · 2026

Read cotomi Act: Learning to Automate Work by Watching You Conference: ACM Conference on AI and Agentic Systems; May 26–29, 2026; San Jose, CA, USA ACM Conference on AI and Agentic Systems (CAIS ’26), May 26–29, 2026, San Jose, CA, USA DOI: 10.1145/3786335.3813203 ISBN: 979-8-4007-2415-2/2026/05 CCS: Computing methodologies Intelligent agents CCS: Human-centered computing Web-based interaction by Masafumi Oyamada, Kunihiro Takeoka, Kosuke Akimoto, Ryoma Obara, Masafumi Enomoto, Haochen Zhang, Daichi Haraguchi, Takuya Tamura (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-03231.txt.

Description

Masafumi Oyamada, Kunihiro Takeoka, Kosuke Akimoto, Ryoma Obara, Masafumi Enomoto, Haochen Zhang, Daichi Haraguchi, and Takuya Tamura present cotomi Act, a browser agent that learns office work by watching its user. The aim is an agent that both runs multi-step web tasks reliably and keeps lasting knowledge of how an organization works.

For execution, lazy observation, compressed history, coarse actions, and best-of-N selection scored above the reported human baseline on a WebArena subset. Separately, passive watching is distilled into task boards and wiki pages in a shared workspace. A controlled test confirmed success rose as that learned knowledge grew.

Catalog record

CREATORMasafumi Oyamada, Kunihiro Takeoka, Kosuke Akimoto, Ryoma Obara, Masafumi Enomoto, Haochen Zhang, Daichi Haraguchi, Takuya Tamura
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE32 KB
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2605.03231

Browse the full library · About the Tower of Babel