SPARTA: Evaluating R easoning S egmentation Robustness through Black-Box A dversarial P araphrasing in T ext A utoencoder Latent Space

Viktoriia Zinkovich, Anton Antonov, Andrei SpiridonovDenis ShepelevAndrey MoskalenkoDaria PugachevaElena TutubalinaAndrey Kuznetsov, Vlad Shakhuro · 2025

Read SPARTA: Evaluating R easoning S egmentation Robustness through Black-Box A dversarial P araphrasing in T ext A utoencoder Latent Space by Viktoriia Zinkovich, Anton Antonov, Andrei SpiridonovDenis ShepelevAndrey MoskalenkoDaria PugachevaElena TutubalinaAndrey Kuznetsov, Vlad Shakhuro (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2510-24446.txt.

Introduction

Viktoriia Zinkovich leads a team of nine authors on AI vision safety. Reasoning segmentation asks a model to outline things named in words. A user might ask for the object that fits a short text clue. Past safety work mostly warped the image to fool the model. The team instead warps the words while keeping the same meaning.

Their method, SPARTA, rewrites a query so the mask gets worse. The rewrite must stay smooth, grammatical, and true to the first sense. A text autoencoder gives a small space where rewrites are searched. Learning from rewards guides that search without seeing model internals. SPARTA beat prior methods by up to two times on two data sets. Even top models proved weak against careful word swaps.

Catalog record

CREATORViktoriia Zinkovich, Anton Antonov, Andrei SpiridonovDenis ShepelevAndrey MoskalenkoDaria PugachevaElena TutubalinaAndrey Kuznetsov, Vlad Shakhuro
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE54 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2510-24446

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

SPARTA: Evaluating R easoning S egmentation Robustness through Black-Box A dversarial P araphrasing in T ext A utoencoder Latent Space Viktoriia Zinkovich, Anton Antonov, Andrei SpiridonovDenis ShepelevAndrey MoskalenkoDaria PugachevaElena TutubalinaAndrey Kuznetsov, Vlad Shakhuro Abstract Multimodal large language models (MLLMs) have shown impressive capabilities in vision-language tasks such as reasoning segmentation , where models generate segmentation masks based on textual queries. While prior work has primarily focused on perturbing image inputs, semantically equivalent textual paraphrases—crucial in real-world applications where users express the same intent in varied ways—remain underexplored. To address this gap, we introduce a novel adversarial paraphrasing task : generating grammatically correct paraphrases that preserve the original query meaning while degrading segmentation performance. To evaluate the quality of adversarial paraphrases, we develop a comprehensive automatic evaluation protocol validated with human studies. Furthermore, we introduce SPARTA —a black-box, sentence-level optimization method that operates in the low-dimensional semantic latent space of a text autoencoder, guided by reinforcement learning. SPARTA achieves significantly higher success rates, outperforming prior methods by up to 𝟐 × \boldsymbol{2\times} on both the ReasonSeg and LLMSeg-40k datasets. We use SPARTA and competitive baselines to assess the robustness of advanced reasoning segmentation models. We reveal that they remain vulnerable to adversarial paraphrasing—even under strict semantic and grammatical constraints. All code and data will be released publicly upon acceptance. 1 Introduction In recent years, foundation models have achieved significant advances across diverse domains of deep learning. Advances in image classification ( Dosovitskiy et al. 2021 ; Liu et al. 2022 ; Woo et al. 2023 ) and interactive segmentation ( Kirillov et al. 2023 ; Ravi et al. 2024 ) , together with progress in large language models (LLMs) ( Brown et al. 2020 ; Touvron et al. 2023 ; Guo et al. 2025 ; Dubey et al. 2024 ) , have paved the way for multimodal large language models (MLLMs) ( Liu et al. 2023 ; Liu et al. 2024a ; Bai et al. 2023 ; Wang et al. 2024b ; Li et al. 2023 ; Peng et al. 2023 ; Lai et al. 2024 ) that seamlessly integrate vision and language. These models are now integral to diverse applications, including conversational systems like ChatGPT, autonomous driving ( Mu et al. 2024 ; Seff et al. 2023 ; Hwang et al. 2024 ) , and robot control ( Driess et al. 2023 ; Brohan et al. 2022 ; Brohan et al. 2023 ; Black et al. 2024 ) . As these models continue to mature, new tasks are emerging—particularly in robotics—that require sophisticated visual perception and reasoning capabilities. One such task is reasoning segmentation ( Lai et al. 2024 ) , where a model outputs a binary segmentation mask driven by an implicit text query that requires intricate logical or contextual interpretation. Figure 1: Example of an adversarial paraphrase generated by our proposed SPARTA method. The SPARTA produces grammatically correct paraphrases that preserve the original semantic content while significantly degrading segmentation performance. The quality of a model’s predicted segmentation mask is expected to remain consistent, even when users paraphrase their prompts, preserving the original meaning and intent. However, the robustness of reasoning segmentation models to query paraphrasing remains largely unexplored. To address this problem, we propose a novel task, adversarial paraphrasing , which constrains textual perturbations according to the following criteria: (1) the core meaning of the original prompt must be preserved; (2) the paraphrase must remain grammatically correct; and (3) it must lead to a degradation in segmentation mask predictions. These constraints enable us to evaluate the robustness of state-of-the-art reasoning-based segmentation models against adversarially paraphrased queries. Based on this task, we construct a new benchmark to systematically evaluate the robustness of reasoning segmentation models. We assess state-of-the-art attack strategies, including gradient-based and LLM-based methods; however, these approaches have notable limitations. Gradient-based methods often produce ungrammatical text ( Guo et al. 2021 ; Jones et al. 2023 ) , while LLM-based attacks typically rely on heuristic methods, such as iterative refinement ( Chao et al. 2024 ) . To overcome the limitations of existing gradient-based and heuristic methods, we introduce SPARTA —a novel black-box sentence-level optimization method (Figure 1 ). SPARTA projects queries into a low-dimensional semantic latent space of a pretrained autoencoder and employs reinforcement learning to identify nearby vectors that yield effective adversarial paraphrases. Overall, our contributions are as follows: • We introduce a novel adversarial paraphrasing task designed to evaluate the robustness of reasoning segmentation models against semantically equivalent paraphrased queries. To facilitate this evaluation, we introduce an automated evaluation protocol. We conduct a user study and demonstrate strong alignment of the proposed scoring method with human judgment. • We present a new method for generating adversarial paraphrases, leveraging reinforcement learning-based sentence-level optimization. Our method outperforms black-box and white-box baselines by up to \boldsymbol{2\times} on both the ReasonSeg and LLMSeg-40k datasets, with 2 model - specific exceptions. • We conduct comprehensive experiments to assess the robustness of state-of-the-art reasoning segmentation models under both white-box and black-box adversarial paraphrasing settings. Our results indicate that, despite the strict semantic and grammatical constraints, existing reasoning segmentation models remain vulnerable to such attacks. 2 Related Work 2.1 Reasoning Segmentation In Referring Expression Segmentation (RES), models output segmentation masks from textual descriptions ( Zou et al. 2023 ; Rasheed et al. 2024 ; Wu et al. 2024b ; Wu et al. 2024a ; Wang et al. 2024c ; Liu et al. 2024b ) . Expanding on RES, the reasoning segmentation task was introduced to handle prompts requiring world knowledge and logical reasoning ( Lai et al. 2024 ) . The pioneering reasoning segmentation model LISA employs an embedding-as-mask paradigm, decoding a <SEG> token via SAM to produce a segmentation mask. Several LISA-based models followed, such as LISA++ ( Yang et al. 2024 ) , which can incorporate segmentation results into text responses, and GSVA ( Xia et al. 2024 ) , which introduces a <REJ> token to explicitly reject absent objects. 2.2 Adversarial Attacks on Text Modality Evaluating model robustness often involves adversarial attacks, which are broadly categorized as white-box or black-box, based on the attacker’s access to model internals. White-box attacks leverage gradient information to optimize adversarial paraphrases, addressing the challenges posed by the discrete nature of text through techniques such as Taylor expansion ( Ebrahimi et al. 2018 ; Jones et al. 2023 ) and Gumbel-Softmax sampling ( Jang et al. 2017 ; Guo et al. 2021 ) . Among these, we consider two state-of-the-art methods: GBDA ( Guo et al. 2021 ) and ARCA ( Jones et al. 2023 ) . While ARCA achieves strong attack success, it lacks semantic regularization, frequently inserting special symbols that undermine its suitability as a paraphrasing baseline. In contrast, GBDA incorporates semantic similarity constraints; however, it remains limited to token-level substitutions, constraining paraphrase diversity. In the black-box setting, attacks have progressed from simple word- and character-level manipulations, such as synonym substitution ( Jin et al. 2020 ; Ren et al. 2019 ) and character edits ( Gao et al. 2018 ) , to methods that generate semantically equivalent paraphrases using transformer-based models ( Li et al. 2020 ; Iyyer et al. 2018 ) . Recent developments further leverage LLMs to generate more

Read the complete text