UTS at PsyDefDetect: Multi-Agent Councils and Absence-Based Reasoning for Defense Mechanism Classification
Dima Galat, Marian-Andrei Rizoiu · 2026
Read UTS at PsyDefDetect: Multi-Agent Councils and Absence-Based Reasoning for Defense Mechanism Classification by Dima Galat, Marian-Andrei Rizoiu (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-09769.txt.
Introduction
Galat and Rizoiu built a system that classifies psychological defense mechanisms in emotional support dialogues. It placed second among 64 teams at the PsyDefDetect task with an F1 score of 0.406. Defense mechanisms are defined by absence: missing affect, blocked thought, denied reality. The team encoded that insight as an affect-cognition spectrum in prompt-level clinical rules, worth the largest single gain of 11.4 points.
Their core engine is a deliberative council of Gemini 2.5 agents, where class advocates rate evidence strength rather than voting, reaching 0.382 with no fine-tuning. The council proved confidently wrong on minority classes, with 59 to 80 percent of stable minority calls incorrect, pulled toward the majority class by an L7 attractor. A targeted override ensemble from three fine-tuned Qwen3.5 models added 2.4 points through just 16 overrides.
Catalog record
| CREATOR | Dima Galat, Marian-Andrei Rizoiu |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 28 KB |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2605.09769
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
UTS at PsyDefDetect: Multi-Agent Councils and Absence-Based Reasoning for Defense Mechanism Classification Dima Galat, Marian-Andrei Rizoiu Abstract This paper describes our system for classifying psychological defense mechanisms in emotional support dialogues using the Defense Mechanism Rating Scales (DMRS), placing second (F1 0.406) among 64 teams. 1 1 1 Code available at https://github.com/dimagalat/bionlp2026 A central insight is that defense mechanisms are defined by what is absent : missing affect, blocked cognition, denied reality. We encode this as an affect-cognition integration spectrum in prompt-level clinical rules, which account for the largest single gain (+11.4pp F1). Our architecture is a multi-phase deliberative council of Gemini 2.5 agents where class-specific advocates rate evidence strength rather than voting, achieving F1 0.382 with no fine-tuning—a top-5 result on its own. We find, however, that the council is confidently wrong about minority classes: 59–80% of stable minority predictions are incorrect, driven by a systematic “L7 attractor” in which emotional content defaults to the majority class. A targeted override ensemble from three fine-tuned Qwen3.5 models applies 16 overrides (+2.4pp), selected by a structured multi-agent system (builder, critic, regression guard) that produced a larger F1 gain in one iteration than 8 prior attempts combined. 1 Introduction The BioNLP 2026 shared task 15 ; 16 requires classifying target utterances in emotional support dialogues into 9 levels of the Defense Mechanism Rating Scales (DMRS; 20 ), a hierarchical clinical instrument ranging from action-based defenses (Level 1) to highly adaptive coping (Level 7). The difficulty is that mechanisms are defined by psychological function , not linguistic form: the same surface expression can indicate denial (Level 3), intellectualization (Level 6), or adaptive coping (Level 7) depending on context (illustrated in § 2 ). The task exhibits severe class imbalance (Table 1 ): Level 7 (Highly Adaptive) comprises 51.9% of training data, while Level 5 and Level 8 account for only 2.6% and 1.5%. We call this the L7 attractor effect : LLMs over-predict the majority class because emotional engagement in therapeutic dialogue looks like “adaptive coping.” Our system uses a multi-phase deliberative council (§ 3.1 ), built on the Gemini 2.5 API 6 : three specialist agents classify in parallel, class-specific advocates rate evidence strength, and a resolution stage adjudicates. We frame the DMRS hierarchy as an affect-cognition integration spectrum: many defense mechanisms are defined by what is absent (missing affect, blocked cognition, denied reality), which requires reasoning about what should be present but is not. A targeted override ensemble from three fine-tuned models applies 16 overrides to the council’s predictions, achieving macro-F1 0.406 (2nd out of 21 registered teams, or 64 CodaBench entries). We also flag a retrieval-leakage risk: same-dialogue exemplars in few-shot prompts inflate validation accuracy from 65% to 97.7% (§ 3.2 ). 2 Task and Data The dataset contains 1,864 training samples from 200 dialogues and 472 test samples from 189 dialogues. All 189 test dialogue IDs overlap with training dialogues (different utterances from the same conversations), creating a retrieval leakage risk addressed in § 3.2 . The official metric is macro-averaged F1; we write “F1” throughout to mean macro-F1. Level Name Train % 0 No Defense / Neutral 296 15.9 1 Action Defense 108 5.8 2 Major Image-Distorting 61 3.3 3 Disavowal 99 5.3 4 Minor Image-Distorting 84 4.5 5 Neurotic Defense 48 2.6 6 Obsessional Defense 172 9.2 7 Highly Adaptive 968 51.9 8 Needs More Information 28 1.5 Table 1: DMRS class distribution in training data. Level 7 accounts for over half of all samples; the five lower-level defense classes (L1–L5) together comprise only 21.5%. The core challenge is illustrated by this training example: a speaker responds to “How are you today?” with “I’m OK. Just dealing with a lot of unknowns.” This reads like Level 7 ( Suppression , consciously managing distress). The ground truth is Level 6 ( Isolation of Affect ): the speaker acknowledges difficulty cognitively (“a lot of unknowns”) but the expected emotional response is absent (“I’m OK”). The distinction turns on what is absent from the utterance, not what is present. 3 System Architecture 3.1 Multi-Phase Deliberative Council Most LLM ensembles aggregate votes. A deliberative council instead evaluates evidence strength per candidate through structured advocacy. In our architecture, Phase 1 agents propose candidates with alternatives (not just top-1 predictions). Phase 2 spawns class-specific advocates that rate fit as strong , moderate , or weak ; each argues for its assigned class using retrieved exemplars. Phase 3 resolves via evidence quality (unique strong wins immediately; ties require pairwise comparison), not vote count. Majority voting loses minority classes because L7 always outnumbers them; evidence-based resolution can select a minority class that receives strong even when the majority favors L7. Figure 1 illustrates the pipeline, which uses the Gemini 2.5 API (primarily Flash, with Pro for resolution) and requires 3–10 LLM calls per sample depending on consensus. Dialogue + Target utterance Clinical Analyst Mechanism Specialist Pattern Analyst Phase 1 Consensus? L i Advoc. L j Advoc. L k Advoc. Phase 2 Resolution: STRONG wins; ties \to pairwise Phase 3 Label (0–8) yes no Figure 1: Council pipeline (3–10 LLM calls per sample). Phase 1: three specialist agents classify in parallel. If unanimous with high confidence, the pipeline exits early (3 calls). Otherwise, Phase 2 spawns class-specific advocates rating fit as strong / moderate / weak (2–5 calls). Phase 3 resolves via priority hierarchy (0–1 call). Formally, given a dialogue d with target utterance u and label space \mathcal{Y}=\{0,\ldots,8\} , the council proceeds in three stages. Three agents a_{1},a_{2},a_{3} each produce a candidate set C_{i}=\{(y_{i},y^{\prime}_{i},p_{i})\} (primary label, alternative, confidence). Let \mathcal{C}=\bigcup_{i}\{y_{i},y^{\prime}_{i}\} be the candidate pool. If all primaries agree ( y_{1}=y_{2}=y_{3}=y^{*} ) with \sum_{i}\mathbf{1}[p_{i}\geq\tau]\geq 2 , the council returns y^{*} immediately. Otherwise, for each unique candidate c\in\mathcal{C} , a class-specific advocate A_{c} produces a strength rating s(c)\in\{\textsc{strong},\textsc{moderate},\textsc{weak}\} . A resolution function R selects the final label via priority ordering: unique strong wins; ties are resolved by pairwise head-to-head comparison. Phase details are in Appendix A . Clinical Knowledge Encoding. We encode the DMRS hierarchy 20 as an affect-cognition integration spectrum . The discriminative question at each level is: what is the relationship between what the speaker knows and what they feel? Cognition present but affect drained \to L6 (Obsessional); affect present but cognition blocked \to L5 (Neurotic); cognition distorts reality to manage affect \to L2–4; affect and cognition integrated \to L7 (Adaptive). The most impactful single test is Reporting vs. Processing (L6/L7): describing painful facts without proportional emotion is Isolation of Affect (L6), not adaptive coping (L7). We complement this with five prompt-level rules: (1) 60+ DMRS-Q behavioral indicators per mechanism 20 ; (2) Emotion \neq Defense : “I feel sad” is not a defense without distortion or transformation; (3) prefer lower-level (less mature) defenses when ambiguous; (4) a watchlist of 8 high-confusion class pairs with discriminative tests; and (5) an L7 verification gate requiring a named adaptive mechanism before permitting L7. In ablation, the Gemini 2.5 Pro council without these clinical rules achieves F1 0.268 (Table 4 ); adding them raises F1 to 0.382 (+11.4pp). Phase-Level Bottleneck. Even with clinical rules, the council’s residual errors concentrate at resolution rather than detection: the correct label enters Phase 2 as a candidate in 96% of errors and