Panoptic Pairwise Distortion Graph
Muhammad Kamran Janjua, Abdul Wahab, Bahador Rashidi · 2026
Read Panoptic Pairwise Distortion Graph by Muhammad Kamran Janjua, Abdul Wahab, Bahador Rashidi (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-11004.txt.
Introduction
Janjua and two colleagues offer a new view of comparison image judgment by showing an image pair through its regions. Existing methods focus on whole-image analysis yet quietly rely on region-level grasp. The authors extend the intra-image scene graph idea to inter-image pairs and propose the distortion graph task. Paired images become a region-grounded graph carrying distortion type, severity, comparison, and quality score. They contribute a region-level dataset called PandaSet, a benchmark suite called PandaBench, and a model called Panda that makes distortion graphs.
Catalog record
| CREATOR | Muhammad Kamran Janjua, Abdul Wahab, Bahador Rashidi |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 70615 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2604.11004
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Panoptic Pairwise Distortion Graph Muhammad Kamran Janjua, Abdul Wahab, Bahador Rashidi Abstract In this work, we introduce a new perspective on comparative image assessment by representing an image pair as a structured composition of its regions. In contrast, existing methods focus on whole image analysis, while implicitly relying on region-level understanding. We extend the intra-image notion of a scene graph to inter-image, and propose a novel task of Distortion Graph ( DG ). DG treats paired images as a structured topology grounded in regions, and represents dense degradation information such as distortion type, severity, comparison and quality score in a compact interpretable graph structure. To realize the task of learning a distortion graph, we contribute (i) a region-level dataset, PandaSet , (ii) a benchmark suite, PandaBench , with varying region-level difficulty, and (iii) an efficient architecture, Panda , to generate distortion graphs. We demonstrate that PandaBench poses a significant challenge for state-of-the-art multimodal large language models (MLLMs) as they fail to understand region-level degradations even when fed with explicit region cues. We show that training on PandaSet or prompting with DG elicits region-wise distortion understanding, opening a new direction for fine-grained, structured pairwise image assessment. Figure 1: DG Task Overview. Top: Given two images, Panda learns the proposed Distortion Graph ( DG ). Bottom: Grounded Subgraphs illustrate how DG grounds regions in terms of distortion information. 1 Introduction In humans, perceptual decisions 1 1 1 any categorical decision about the presence or identity of a sensory stimulus are often cognitively involved, deliberate, and contextual ( 14 ) . Studies have argued that any model of such perceptual decision making should consider the representation of the relevant sensory input and how that representation is formed ( 18 ) . In the case of visual stimuli, one example of such perceptual decisions is distortion analysis. Yet, when it comes to computational perceptual decision making, often, current design choices favor a top-down approach considering a global view of the input for analyses tasks ( 70 ; 31 ; 59 ; 65 ; 58 ) . Figure 2: Motivation. Current MLLMs (e.g., Co-Instruct ( 59 ) ) fail at region-level understanding, struggling even when given explicit region details (name, description, bounding box). DG grounds assessment in regions, relating distortions and attributes to provide a structured view. Optionally, the graph can be fed to an MLLM for region-wise language descriptions. Scene relations (yellow) are not predicted. Best viewed zoomed-in. Such design choices are inherently limiting because they do not lend themselves to fine-grained understanding ( 44 ) . Further, in multimodal language models (MLLMs), they restrict distortion-specific visual question answering (VQA), ranking, descriptive understanding, and even quality scoring to image-level ( 31 ; 26 ; 65 ) . With instruction tuning ( 36 ) on a limited instruction set as the learning paradigm, more often than not, the outcome is a rigid multi-billion parameter model that parrots template responses ( 67 ; 12 ) , see fig. 2 . One reason for the prevalence of a top-down approach is the lack of a structured representation that is grounded in image regions. In this work, we offer a novel perspective on learning a structured representation between image pairs for assessment, and introduce the task of a Distortion Graph ( DG ) . DG is a general-purpose pairwise graph structure, with regions as atomic components. Each node corresponds to a region, while inter-region edges capture comparative relationships (predicates). Nodes encode the local distortion and severity type as well as a region-level quality score (attributes), enabling region-first reasoning over paired images, see fig. 1 . We argue that DG is a pertinent structured approach for pairwise comparative purposes since such information aggregates to image-level judgments, while vice-versa is often not true. We position DG such that it can complement MLLMs in offering region-wise distortion analysis in natural language, see fig. 2 for illustration. To realize the task of learning a distortion graph, we introduce a region-level distortion dataset, termed as PandaSet . By design, PandaSet comprises over 500 K image pairs degraded by 15 different distortions, ranging from sensor-induced and equipment failure distortions to weather distortions, with four different severity levels. Each region has an associated quality score indicating on what end of the distortion spectrum (from clean to severely degraded) it lies. We show that the proposed task is indeed computationally tractable and design an efficient architecture, termed Panda , that learns to predict region-level attributes and predicates to generate DG . We introduce PandaBench , a benchmark derived from PandaSet with three splits of increasing region-level difficulty to enable systematic evaluation. We evaluate both open-source and closed-source frontier MLLMs on the proposed benchmark under zero-shot and fine-tuned setups. We empirically show that distortion-specific MLLMs suffer greatly when reasoning over image regions and often resort to template responses, even when explicitly prompted with region-wise visual markers. Further, such methods are limited by the context length in terms of processing variable number of regions. On the other hand, frontier MLLMs are less rigid and have superior instruction following abilities, yet their performance is not much better than random chance. Additionally, as a showcase application, we demonstrate that DG in chain-of-thought prompting encourages emergent capabilities of MLLMs for distortion understanding. 2 Related Work Distortion MLLMs. One of the earliest efforts towards enabling low-level vision understanding in MLLMs is Q-Instruct, wherein 57 introduced a new dataset, Q-Pathways, and instruction-tuned LLaVA-v1.5 ( 34 ) for distortion identification and VQA. Several works followed suit, introducing improved benchmarks, training recipes, and methods. 70 proposed an extension to Q-Pathways, and unified the task of image quality assessment in terms of numerical scores and descriptive analysis in MLLMs. 56 introduced a benchmark Q-Bench comprising low-level attribute and descriptive tasks, along with quality score regression to evaluate MLLMs’ ability on low-level vision. However, all of these works focus on single image analysis. Towards comparative assessment, 65 proposed a new dataset, M-BAPPS, and fine-tuned Vicuna-v1.5 ( 10 ) on quality comparison task in the full-reference setting (i.e., a clean reference image should be available). DepictQA ( 64 ) introduced a general-purpose dataset, DQ495K, to let MLLMs perform similar comparative tasks but even without any reference image. Co-Instruct ( 59 ) introduced MICBench, a benchmark to evaluate MLLMs performance on comparative tasks, but the introduced method allowed multiple images to be compared. Note that specialist models in pre-MLLM era often focused on learning a numerical score for image quality assessment (IQA), and several works exist in literature along this direction ( 5 ; 1 ; 52 ) . Region Understanding. None of the above mentioned work is region-first by design. Further, the MLLMs fine-tuned on these datasets can not extend to regions since that is an out-of-domain setting ( 45 ) , also see fig. 2 . Literature, however, has made efforts to enable general-purpose region-level understanding in MLLMs. Set-of-Mark (SoM) prompting ( 61 ) introduced visual markers overlaid on regions to prompt MLLMs to reason about regions. 53 proposed to utilize region-of-interest (RoI) features to generate region-level tokens for MLLMs. Omni-RGPT ( 22 ) introduced token markers for region-level comprehension in images and videos. One particular work, Seagull ( 8 ) , explored region-level descriptive distortion analysis,