Taming Scylla: Understanding the multi-headed agentic daemon of the coding seas

Micah Villmow · 2026

Read Taming Scylla: Understanding the multi-headed agentic daemon of the coding seas by Micah Villmow (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-08765.txt.

Description

Coding agents are proliferating, but there is no rigorous way to compare how prompts, skills, tools, and multi-agent setups affect capability and cost. Micah Villmow introduces Scylla, an evaluation framework based on structured ablation studies across seven testing tiers of rising complexity. His arXiv paper centers on Cost-of-Pass: the expected dollar cost to obtain one correct solution.

The framework is model-agnostic and works with any command-line tool, demonstrated with Claude Sonnet 4.5 and judged by multiple models from the same vendor using direct tests, rubrics, and qualitative assessment. Results suggest architectural complexity does not always improve quality. Readers get a reproducible way to price agent designs against their actual outcomes.

Catalog record

CREATORMicah Villmow
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE62 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2602-08765

Browse the full library · About the Tower of Babel