FormalJudge : A Neuro-Symbolic Paradigm for Agentic Oversight

Jiayi Zhou, Yang Sheng, Hantao Lou, Yaodong Yang, Jie Fu · 2026

Read FormalJudge : A Neuro-Symbolic Paradigm for Agentic Oversight by Jiayi Zhou, Yang Sheng, Hantao Lou, Yaodong Yang, Jie Fu (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-11136.txt.

Description

Who watches the watchers when judges are themselves chance-based? Jiayi Zhou and four co-authors propose FormalJudge, a neuro-symbolic frame for agentic oversight. Language models first turn high-level intent into small, checkable constraints, then prove compliance with Dafny specs and a satisfiability solver.

That two-way Formal-of-Thought design replaces chance-based scores with math guarantees. Tests span behavioural safety, multi-domain constraint following, and deception finding. The frame beats language-model-judge rivals by a clear margin, lets a small judge catch deception from far larger agents, and gains steadily with iterative refinement. The work argues that formal checks can escape the judge's own failure modes.

Catalog record

CREATORJiayi Zhou, Yang Sheng, Hantao Lou, Yaodong Yang, Jie Fu
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE73 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2602-11136

Browse the full library · About the Tower of Babel