Editor’s Choice : Evaluating Abstract Intent in Image Editing through Atomic Entity Analysis

Mor Ventura, Roy Hirsch, Yonatan Bitton, Regev Cohen, Roi Reichart · 2026

Read Editor’s Choice : Evaluating Abstract Intent in Image Editing through Atomic Entity Analysis by Mor Ventura, Roy Hirsch, Yonatan Bitton, Regev Cohen, Roi Reichart (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-14842.txt.

Introduction

Ventura, Hirsch, Bitton, Cohen and Reichart formalise abstract image editing, where people communicate through concepts like mood rather than literal commands. Current benchmarks focus on explicit instructions, leaving abstract instructions underexplored. The authors introduce Entity-Rubrics, a framework splitting abstract edits into entity-level assessments that correlate strongly with human judgement. Alongside it comes AbstractEdit, the first benchmark for abstract image editing across diverse real scenes. Eleven leading models struggle to balance intent and preservation, defaulting to under-editing or over-editing.

Catalog record

CREATORMor Ventura, Roy Hirsch, Yonatan Bitton, Regev Cohen, Roi Reichart
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE91814
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2605.14842

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Editor’s Choice : Evaluating Abstract Intent in Image Editing through Atomic Entity Analysis Mor Ventura, Roy Hirsch, Yonatan Bitton, Regev Cohen, Roi Reichart Abstract Humans naturally communicate through abstract concepts like “mood”. However, current image editing benchmarks focus primarily on explicit, literal commands, leaving abstract instructions largely underexplored. In this work, we first formalize the definition and taxonomy of abstract image editing. To measure instruction-following in this challenging domain, we introduce Entity-Rubrics , a framework that breaks down abstract edits into individual, entity-level assessments and achieves strong correlation with human judgment. Alongside this framework, we contribute AbstractEdit , the first benchmark dedicated to abstract image editing across diverse real-world scenes. Evaluating 11 leading models on this dataset reveals a fundamental challenge: standard architectures struggle to balance intent and preservation, commonly defaulting to under-editing or over-editing. Our analysis demonstrates that driving meaningful improvements relies heavily on integrating advanced LLM text encoders and iterative thinking. Looking forward, our entity-based paradigm can generalize beyond assessment to serve as a reward model, enable models to correctly interpret abstract communication, or highlight specific failures in test-time critique loops. Ultimately, we hope this work serves as a stepping stone toward seamless multimodal interaction, closing the gap between rigid machine execution and the natural, open-ended way humans communicate. 1 1 1 Code and data available via the project page: https://venturamor.github.io/EditorsChoice/ . Figure 1: Evaluating Abstract Image Editing . Given a context image and an abstract instruction from our AbstractEdit test set, our Entity-Rubrics framework assesses the edited image by decomposing the scene at a granular level, yielding a per-edit rank that determines the final instruction-following score. As shown, some entity edits lead to better prompt alignment (e.g., the man’s expression), while others introduce unnecessary changes that hurt preservation (the crowd). 1 Introduction Image generation models have made significant progress in following explicit commands ( brooks2023instructpix2pix ; betker2023improving ) , yet their ability to interpret abstract instructions remains a critical bottleneck ( liu2022design ; fu2023guiding ) . In creative applications, humans rarely think in terms of exhaustive, technical specifications; instead, they communicate through high-level concepts, such as altering an image’s mood or season ( norman2014some ; zamfirescu2023johnny ; oppenlaender2024taxonomy ) . Natural communication relies on a shared understanding of the world, allowing humans to convey complex ideas without specifying every granular detail ( grice1975logic ; clark1991grounding ) . However, current visual generative models still struggle to achieve this level of abstract semantic alignment. Prior work has recognized this issue but has typically approached it from narrow angles (§ 2 ). Studies have predominantly focused on limited implicit instructions ( pu2025picabench ; zeng2025editworld ; yu2025anyedit ) , or on physical reasoning ( he2025reasoning ; zeng2025editworld ; yu2025anyedit ) , where target outcomes are singular and easily quantifiable, leaving little room for true abstraction. This hesitancy to fully tackle abstract instructions in real-world scenes stems from their inherent difficulty: abstract prompts are highly subjective, allowing many valid interpretations. Therefore, in this work, we establish a framework to tackle the broader challenge of abstract instruction following through a formal problem definition, and a novel evaluation suite . First, we argue that instruction-based image editing provides a uniquely highly practical testbed for bridging this gap compared to a pure text-to-image setup. Generally, image editing is considered more complex than text-to-image generation due to a dual objective: the model must adhere to the text instruction while rigorously preserving the original image context. However, when evaluating abstract instructions, this dual constraint is actually a profound advantage. The requirement to preserve the context image significantly narrows the infinite space of possible interpretations, anchoring the abstract request to a grounded visual baseline and making systematic evaluation tractable. Next, we define abstract image editing (§ 3 ). It is determined by both the text instruction and the context image, and is measured along two axes spanning human intent: Identification and Specificity. This concept can be conveyed by asking a simple question: would a majority of human observers identify the exact same target entities to be edited, and would they expect the exact same visual transformations? If either of these conditions fails, the prompt is considered an abstract edit. While explicit instructions, such as “add mud to the paws” provide a clear roadmap, the intent in Fig. 2 to “make the dog look like it just returned from a long rainy trip” is entirely abstract; specifying neither target nor transformation. Such abstract intent allows diverse valid edits, such as making the fur wet, moving to a bathtub, or adding an umbrella, requiring the model to autonomously deduce multiple required edits. This definition of abstractness as a subjective, multi-path problem naturally dictates a shift toward granularity and precision in the evaluation. To achieve this granularity, we draw inspiration from factuality evaluation in NLP. In that domain, complex generated text is assessed by decomposing it into “atomic facts” and verifying each one against a source document ( gekhman2023trueteacher ; min2023factscore ; honovich2022true ; wei2024long ) . We posit that abstract visual transformations can be similarly evaluated by analyzing the discrete modifications applied to specific image entities. Despite the subjectivity of what and how to edit, results must be objectively grounded in the input narrative. Since holistic transformations are more than a “sum of parts” we treat entity-level consistency as a necessary, but not sufficient, requirement for overall alignment. Operationalizing this perspective, we propose Entity-Rubrics , a VLM-based, novel granular automatic evaluation framework. Entity-Rubrics identifies and treats individual image entities, such as objects and stuff, as the atomic units of assessment (§ 4 ), and provides a highly interpretable rationale, as opposed to existing evaluation methods which at most provide only a single global score and a short explanation ( ku2024viescore ; yang2025texttt ) . Since abstract prompts allow for multiple valid interpretations, target edits are not known a priori. We therefore prioritize precision over recall: rather than checking if a model captured every possible way to interpret a prompt, we verify that the specific edits it chose are logically grounded in the prompt. By that, our approach captures the multiple local edits triggered by abstract instruction, and distinguishes intent-driven transformations from arbitrary alterations that hurt the context image preservation (Fig. 1 ), yielding a strong correlation with human judgment. Figure 2: Taxonomy of Image Editing Intent . We formalize intent across two orthogonal axes: Identification (“what to edit”) and Specificity (“how to edit”). Intent is categorized based on the mapping between these axes: Explicit and Implicit editing provide a one-to-one mapping through high consensus on both axes, whereas abstract editing introduces a one-to-many relationship. To address the lack of structured resources, we introduce AbstractEdit ( § 5 ), a diverse dataset of editing scenarios designed specifically to evaluate abstract intent. The test set consists of 470 human-verified abstract samples spanning four core domains, ranging from physical and logical changes to emotional and social shifts. Each sample is a triplet consisting of a context image, an abstract edit instruction, and a corresponding explicit instruction serving as a representative concrete example. To support model development, we further provide a training expansion of 4k samples. AbstractEdit is produced via a novel, fully automated VLM-powered pipeline designed to ensure the systematic generation of contextually

Read the complete text