PATHWAYS: Evaluating Investigation and Context Discovery in AI Web Agents

Shifat E. Arman, Syed Nazmus Sakib, Tapodhir Karmakar Taton, Nafiul Haque, Shahrear Bin Amin · 2026

Read PATHWAYS: Evaluating Investigation and Context Discovery in AI Web Agents by Shifat E. Arman, Syed Nazmus Sakib, Tapodhir Karmakar Taton, Nafiul Haque, Shahrear Bin Amin (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-05354.txt.

Description

Web agents sent to moderate content or enforce policy often face a trap. The visible page suggests one action, while the decisive evidence is buried a few clicks away. Syed Nazmus Sakib and 4 co-authors built a benchmark for that investigative skill in this 2026 preprint, running 39 pages. The hosted title calls the suite PATHWAYS; inside, the benchmark is named MIRAGE and holds 750 multi-step decision tasks.

Tasks span Wikipedia forensics, shopping administration, and Reddit moderation, each with a misleading surface layer and a hidden layer reachable only by active digging. Scoring splits investigation, reasoning, and final decision accuracy, plus a rate of fabricated citations. Eight agents across two model generations showed three stubborn patterns. They reach the right pages but miss the decisive evidence. Hints help them search without reliably fixing decisions, and about one trajectory in eight cites facts that were invented.

Catalog record

CREATORShifat E. Arman, Syed Nazmus Sakib, Tapodhir Karmakar Taton, Nafiul Haque, Shahrear Bin Amin
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE88 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2602-05354

Browse the full library · About the Tower of Babel