GRASP: Deterministic argument ranking in interaction graphs

Diganta Misra, Antonio Orvieto, Rediet Abebe, Volkan Cevher · 2026

Read GRASP: Deterministic argument ranking in interaction graphs by Diganta Misra, Antonio Orvieto, Rediet Abebe, Volkan Cevher (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-19141.txt.

Introduction

Diganta Misra and three colleagues show that holistic debate judging by language models, giving one verdict, suffers wide disagreement between models. They trace the instability to collapsing interaction structure into a single opaque score. Their remedy is GRASP, a deterministic framework aggregating local judgments about attacks and support into a global ranking via a propagation operator. Local judgments prove more reproducible than holistic rankings, so GRASP yields steadier global rankings. Its scores do not track human convincingness labels, showing GRASP measures structural sufficiency, not persuasion or rhetorical appeal.

Catalog record

CREATORDiganta Misra, Antonio Orvieto, Rediet Abebe, Volkan Cevher
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE105827
LICENSEhttp://creativecommons.org/licenses/by-sa/4.0/
RIGHTSCC BY SA
SOURCEarXiv

Source: https://arxiv.org/abs/2605.19141

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

GRASP: Deterministic argument ranking in interaction graphs Diganta Misra, Antonio Orvieto, Rediet Abebe, Volkan Cevher Abstract Large language models are increasingly deployed as automated judges to evaluate the strength of arguments. As this role expands, their legitimacy depends on consistency, transparency, and the ability to separate argumentative structure from rhetorical appeal. However, we show that holistic judging—a common LLM-as-a-Judge practice where a model provides a global verdict on a debate—suffers from substantial inter-model disagreement. We argue that this instability arises from collapsing a debate’s complex interaction structure into a single opaque score. To address this, we propose GRASP (Gradual Ranking with Attacks and Support Propagation), a deterministic framework that aggregates stable local interaction judgments into a global ranking via a convergent attack–defense propagation operator. We show that local interaction judgments are more reproducible than holistic rankings in LLM-as-a-Judge evaluations, allowing GRASP to produce more consistent global rankings. We further show that GRASP scores do not correlate with human “convincingness” labels, highlighting a vital sociotechnical distinction: GRASP does not measure persuasion, factuality, or rhetorical appeal, but structural sufficiency—a defense-aware notion of argument robustness over the explicit interaction graph. Overall, GRASP offers a transparent and auditable alternative to holistic LLM judging. 1 Introduction Large language models (LLMs) are increasingly used not only to generate content, but also to act as automated judges for evaluating discourse quality, moderating debates [ 22 ] , and supporting multi-agent decision-making [ 5 , 50 , 1 , 37 ] . For such systems to be credible arbiters of deliberation, their evaluations must be consistent, transparent, and grounded in explicit argumentative structure rather than model-specific idiosyncrasies. Most current practice relies on holistic judging , where an LLM is presented with an entire debate and asked to output a global verdict or ranking [ 68 , 37 , 60 ] . We show that these global judgments exhibit substantial inter-model disagreement, consistent with growing concerns about the reliability of LLM-as-a-Judge paradigms [ 66 , 20 ] . Such instability suggests that holistic judging may conflate argumentative structure with rhetorical and stylistic preferences—e.g., biases, verbosity, tone, or narrative coherence [ 59 , 58 , 32 ] . We argue that this reflects a structural limitation of the paradigm: global judgments collapse rich dialectical interactions into a single black-box score. Instead, we ground evaluation in local semantic interactions , drawing on computational argumentation and abstract argumentation frameworks [ 25 , 11 , 9 , 62 ] . While models may disagree on holistic verdicts, they are markedly more consistent on local pairwise judgments [ 46 , 14 , 40 ] , making such interactions a more reliable primitive. We therefore ask: Research question. Can argument ranking be grounded in explicit local interaction structure rather than opaque holistic LLM-as-a-Judge verdicts? To address this question, we introduce GRASP ( G radual R anking with A ttacks and S upport P ropagation), a convergent propagation algorithm that composes local attack and support judgments into a global ranking by aggregating direct attacks and higher-order defenses on an explicit interaction graph. This yields a notion of argument strength tied to the explicit interaction graph rather than to an argument’s truth, persuasiveness, rhetorical quality, or human preference. We formalize this target as structural sufficiency : a defense-aware notion of argument robustness defined relative to an explicit interaction graph. Structural sufficiency is closest to global sufficiency [ 23 , 31 ] , but evaluates robustness only with respect to the instantiated structure, rather than all counterarguments that could be anticipated. It is related to gradual and ranking-based argumentation semantics [ 7 , 2 ] . Empirically, we show that GRASP produces substantially more reproducible rankings than direct LLM judging, while its weak correlation with human “convincingness” labels supports the distinction between structural sufficiency and persuasive effectiveness. Contributions. • We document substantial inter-model disagreement in holistic LLM-based judging and show that local interaction judgments are more reproducible (§ 5 , Appendix F.5 ). • We introduce GRASP , a convergent propagation algorithm for structural argument ranking over explicit interaction graphs (§ 2 , § 3 ). • We formalize structural sufficiency , a defense-aware notion of argument robustness over explicit interaction graphs (§ 4 ). • We show that GRASP -based rankings are highly reproducible across models while capturing a notion of robustness distinct from human convincingness (§ 5 , Appendix L ). 2 GRASP: A Structural Strength Propagation Operator Motivated by the unreliability of holistic argument evaluation and the relative consistency of local relational judgments, we introduce GRASP (Gradual Ranking with Attacks and Support Propagation), an iterative operator that computes continuous argument strengths from an explicit interaction graph. GRASP aggregates local attack and defense relations into a global ranking while remaining agnostic to rhetorical appeal, persuasion, or hypothetical objections. 2.1 Background: Abstract Argumentation and Ranking Semantics We build on Abstract Argumentation Frameworks (AAFs) [ 25 ] , where arguments are nodes and attacks are directed edges. Classical AAF semantics focus on set-valued notions of acceptability, such as admissible, preferred, stable, and grounded extensions [ 13 , 7 ] , which are ill-suited for fine-grained comparison among individual arguments. Ranking-based semantics address this limitation by assigning each argument a numerical or ordinal strength [ 2 ] . A prominent example is the H-categorizer [ 11 ] , which penalizes arguments according to the total strength of their attackers. While effective as a local heuristic, such methods treat attacks independently and do not explicitly model how arguments may be defended by other arguments. Our aim is to retain the simplicity and interpretability of attack-based rankings while incorporating defense through an explicit propagation rule. 2.2 Weighted Interaction Graphs Let A=\{a_{1},\dots,a_{n}\} be a set of arguments. GRASP operates on a weighted interaction graph \mathcal{G}=(A,W,D) , with attack matrix W\in[0,1]^{n\times n} and defense matrix D\in[0,\infty)^{n\times n} . Here, W_{ij} is the strength with which a_{i} attacks a_{j} , and D_{kj} is the extent to which a_{k} defends a_{j} . 2.3 The GRASP Update Rule Given an initial strength vector \mathbf{s}^{(0)}\in\mathbb{R}_{\geq 0}^{n} , GRASP iteratively updates argument strengths through a nonlinear operator that balances the weakening effect of attacks against the reinforcing effect of defense. Undamped operator. We first define the undamped GRASP operator G:\mathbb{R}^{n}\to\mathbb{R}^{n} coordinatewise G(s)_{j}\;=\;\frac{1+\beta\sum_{k}D_{kj}s_{k}}{1+\alpha\sum_{i}W_{ij}s_{i}}, (1) where the denominator penalizes argument a_{j} according to the total strength of its attackers, while the numerator rewards it according to the total strength of its defenders. The parameters \alpha,\beta\geq 0 control the relative influence of attack and defense. GRASP Update Rule Define the damped operator \widehat{G}:\mathbb{R}^{n}\to\mathbb{R}^{n} : \widehat{G}(s)=(1-\gamma)s+\gamma\,G(s) (2) The iteration is then given by: s^{(t)}=\widehat{G}\bigl(s^{(t-1)}\bigr) (3) or coordinatewise: s_{j}^{(t)}=(1-\gamma)s_{j}^{(t-1)}+\gamma\frac{1+\beta\sum_{k}D_{kj}s_{k}^{(t-1)}}{1+\alpha\sum_{i}W_{ij}s_{i}^{(t-1)}} (4) Damped GRASP operator and iteration. To improve numerical stability and guarantee convergence in dense or highly cyclic interaction structures, we employ a damped version of the operator, analogous to relaxation schemes in iterative optimization. Repeated application yields stable argument strengths reflecting how well each argument withstands attack within the explicit interaction structure. Final rankings order arguments by converged strength. 2.4 Interpretation GRASP generalizes attack-based ranking methods such as the H-categorizer by explicitly incorporating defense while preserving locality and interpretability. The resulting scores operationalize the notion of structural sufficiency introduced in Section 4 : arguments are strong to the extent that incoming attacks are

Read the complete text