ObjexMT: Objective Extraction and Metacognitive Calibration for LLM - as - a - Judge under Multi - Turn Jailbreaks

Hyunjun Kim, Junwoo Ha, Sangyoon Yu, Haon Park, [1em] AIM Intelligence, University of Seoul, Korea Advanced Institute of Science and Technology, Seoul National University [email protected] , [email protected] , [email protected] , [email protected] · 2025

Read ObjexMT: Objective Extraction and Metacognitive Calibration for LLM - as - a - Judge under Multi - Turn Jailbreaks by Hyunjun Kim, Junwoo Ha, Sangyoon Yu, Haon Park, [1em] AIM Intelligence, University of Seoul, Korea Advanced Institute of Science and Technology, Seoul National University [email protected] , [email protected] , [email protected] , [email protected] (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2508-16889.txt.

Introduction

Hyunjun Kim and three co-authors test whether AI judges can name the true aim of a chat and know when to trust that call. Models that grade other models are prized for scale, yet no sharp test of this skill existed. Long chats and jailbreak tricks scatter the true aim across turns, which trips most judges. Their test set feeds a model a full multi-turn chat and asks for a one-line aim plus a trust score.

Scores weigh how close the aim line sits to the gold line, with a cut set on a warm-up set. Trust scores are graded on how well high trust lines up with true hits. Six top models ran three attack chat sets. The best aim score and the best trust score sat in distinct models. Auto fakes hurt far more than model choice did.

Catalog record

CREATORHyunjun Kim, Junwoo Ha, Sangyoon Yu, Haon Park, [1em] AIM Intelligence, University of Seoul, Korea Advanced Institute of Science and Technology, Seoul National University [email protected] , [email protected] , [email protected] , [email protected]
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE36 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2508-16889

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

ObjexMT: Objective Extraction and Metacognitive Calibration for LLM - as - a - Judge under Multi - Turn Jailbreaks Hyunjun Kim, Junwoo Ha, Sangyoon Yu, Haon Park, [1em] AIM Intelligence, University of Seoul, Korea Advanced Institute of Science and Technology, Seoul National University [email protected] , [email protected] , [email protected] , [email protected] Abstract LLM-as-a-Judge (LLMaaJ) now underpins scalable evaluation, yet we lack a decisive test of a judge’s qualification: can it recover a conversation’s latent objective and know when that inference is trustworthy? LLMs degrade under irrelevant or long context; multi-turn jailbreaks further hide goals across turns. We introduce ObjexMT , a benchmark for objective extraction and metacognition. Given a multi-turn transcript, a model must return a one-sentence base objective and a self-reported confidence. Accuracy is computed via LLM-judge semantic similarity to gold objectives, converted to binary correctness by a single human-aligned threshold calibrated once on N=100 items ( τ ⋆ = 0.61 \tau^{\star}\!=\!0.61 ). Metacognition is evaluated with ECE, Brier, Wrong@High-Conf, and risk–coverage. Across gpt-4.1 , claude-sonnet-4 , and Qwen3-235B-A22B-FP8 on SafeMTData_Attack600 , SafeMTData_1K , MHJ , and CoSafe , claude-sonnet-4 attains the best objective-extraction accuracy (0.515) and calibration (ECE 0.296; Brier 0.324); gpt-4.1 and Qwen3-235B-A22B-FP8 tie at 0.441 but are overconfident (mean confidence ≈ \approx 0.88 vs. accuracy ≈ \approx 0.44; [email protected] ≈ \approx 48–52%). Performance varies by dataset ( ≈ \approx 0.167–0.865). ObjexMT thus supplies an actionable test for LLM judges: when objectives are not explicit, judges often misinfer them with high confidence. We recommend exposing objectives when feasible and gating decisions by confidence otherwise. Code and data at https://github.com/hyunjun1121/ObjexMT_dataset . 1 Introduction From scalable evaluation to objective understanding. Large language models (LLMs) have rapidly become not only subjects of evaluation but also instruments of evaluation. The “LLM-as-a-Judge” (LLMaaJ) paradigm promises scalable, low-latency assessment across tasks, domains, and model variants, and is increasingly used to replace or triage human raters [ 3 ] . Yet the community lacks a precise answer to a fundamental question that bears directly on when—and how—LLM judges should be trusted: can an LLM reliably infer the latent objective of a prompt or conversation it is asked to evaluate? This question is not academic. Real deployments frequently involve multi-step, noisy exchanges where the user’s ultimate goal is not stated verbatim. Why multi-turn jailbreaks are the hardest case. If there exists a setting where objective understanding is maximally stressed, it is multi-turn jailbreak prompting. Contemporary red-teaming shows that adversaries distribute or disguise a harmful goal over several turns, interleaving distractors, role-play wrappers, and coreference so that the true objective is plausibly deniable or temporally distant [ 6 , 15 , 9 ] . This structure exacerbates well-documented weaknesses of LLMs under irrelevant context and long inputs—e.g., vulnerability to off-topic sentences [ 10 ] , position-dependent recall failures such as the “lost-in-the-middle” effect [ 8 ] , and distractor insertion robustness gaps [ 7 ] . In short, multi-turn jailbreaks form a rigorous stress test of whether an LLM judge can recover the disguised intent , not merely label a surface string as (un)safe. Discriminating harmfulness is not the same as inferring intent. Recent findings suggest LLMs can excel as harmfulness discriminators while remaining comparatively fragile as generators when directly processing adversarial inputs—revealing a detection–generation discrepancy [ 1 ] . However, classifying overt harmfulness and inferring a hidden objective from a noisy, multi-turn transcript are different cognitive demands. We empirically find that state-of-the-art LLMs often fail to pinpoint the true objective of multi-turn jailbreaks and, troublingly, frequently assign high self-reported confidence to incorrect extractions. This gap challenges the assumption that an LLM judge can safely self-supply the missing objective during evaluation. Why metacognition (confidence) matters for LLM-as-judge. Because LLMs are opaque predictors, a judge must signal when its own judgment is trustworthy. We therefore treat the model’s self-reported confidence as a proxy for metacognition —i.e., knowing what it knows and what it does not. Prior work shows that (i) LLMs’ expressed or “verbalized” confidence can be elicited and, in some cases, made more truthful than raw token probabilities [ 12 ] , (ii) models exhibit varying degrees of self-knowledge in recognizing unanswerable queries [ 14 ] , and (iii) calibration of confidence is an active area with established metrics such as Expected Calibration Error (ECE), Brier score, and selective-prediction curves [ 2 , 5 ] . In brief, an LLM suitable for judging others must not only label outputs but also calibrate its certainty: high confidence when correct, low confidence when likely wrong. This paper: ObjexMT. We introduce ObjexMT (Objective EXtraction in Multi-Turn jailbreaks), a benchmark that isolates and measures an LLM’s ability to (i) recover the base objective of a multi-turn jailbreak conversation and (ii) calibrate its confidence in that recovery. We evaluate on SafeMTData_Attack600 and SafeMTData_1K (collectively, SafeMTData), MHJ (human multi-turn jailbreaks), and CoSafe [ 9 , 6 , 15 ] . Concretely, a model receives a multi-turn transcript and must return a one-sentence base prompt that plainly states the attacker’s goal, together with a confidence score in [0,1] . For accuracy, we use an LLM evaluator to assign a semantic similarity score between the extracted and gold objectives; for metacognition , we analyze how confidence aligns with correctness via reliability diagrams (ECE), Brier score, Wrong@High-Conf, and selective-prediction trade-offs. Contributions. • Problem framing. We formalize objective extraction under multi-turn jailbreaks across SafeMTData_Attack600 , SafeMTData_1K , MHJ , and CoSafe [ 9 , 6 , 15 ] . • Benchmark, metrics, and metacognition. We release ObjexMT at https://github.com/hyunjun1121/ObjexMT_dataset with: (i) instruction and scoring templates, (ii) the complete experimental dataset (raw transcripts and I/O, extracted objectives and confidences, judge similarity scores and categories, human calibration labels, and per-row spreadsheets), and (iii) materials to support reproducible analysis. Evaluation combines LLM-based semantic matching with metacognitive calibration analyses (ECE, Brier, Wrong@High-Conf, and selective prediction) to quantify whether models know when they know . • Findings and guidance for LLM-as-judge. Across models, objective extraction is error-prone and often overconfident , contrasting with reports that LLMs can reliably detect harmful prompts [ 1 ] . We discuss when LLM judges should be provided the objective explicitly during evaluation, supported by evidence from recent evaluators that do expose the objective and achieve strong alignment with human judgments [ 4 , 11 ] . Broader impact. We position ObjexMT as a general diagnostic for objective understanding and metacognitive reliability under noisy multi-step inputs, with immediate ramifications for safety evaluation. By starting from a general notion of latent-intent recovery and then specializing to multi-turn jailbreaks, our results invite reuse beyond safety (e.g., multi-hop QA and tool-use auditing) while delivering concrete prescriptions for LLM safety evaluators. 2 Related Work 2.1 LLM-as-a-Judge LLM-based evaluators are increasingly used to scale benchmarking and moderation workflows, offering flexibility across tasks, languages, and criteria [ 3 ] . Despite efficiency gains, the community continues to probe their reliability, bias, and failure modes, particularly when key task information is not stated explicitly and must be inferred from context. 2.2 Robustness under irrelevant and long context A growing body of work shows LLMs are brittle under seemingly innocuous context changes. Adding irrelevant sentences can sharply reduce accuracy across prompting strategies [ 10 ] ; long-context usage exhibits strong positional effects, with precipitous drops

Read the complete text