When to Call an Apple Red: Humans Follow Introspective Rules, VLMs Don’t

Jonathan Nemitz, Carsten Eickhoff, Junyi Jessy Li, Kyle Mahowald, Michal Golovanevsky & William Rudman · 2026

Read When to Call an Apple Red: Humans Follow Introspective Rules, VLMs Don’t by Jonathan Nemitz, Carsten Eickhoff, Junyi Jessy Li, Kyle Mahowald, Michal Golovanevsky & William Rudman (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-06422.txt.

Introduction

Nemitz and five others introduce the Graded Color Attribution dataset. Line drawings vary color coverage across dyes and no-prior shapes. Models and humans state a threshold pixel share for labels. Rules face later calls. Models violate their own rules: GPT-5-mini violates its stated rules in nearly 60 percent of strong-prior cases. Humans remain faithful, seeming breaks from high guesses of coverage. Models estimate cover well, yet fight their thought. World-knowledge priors degrade faithfulness. The findings challenge hardness-driven views, hinting introspective self-knowledge is miscalibrated.

Catalog record

CREATORJonathan Nemitz, Carsten Eickhoff, Junyi Jessy Li, Kyle Mahowald, Michal Golovanevsky & William Rudman
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE47357
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2604.06422

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

When to Call an Apple Red: Humans Follow Introspective Rules, VLMs Don’t Jonathan Nemitz, Carsten Eickhoff, Junyi Jessy Li, Kyle Mahowald, Michal Golovanevsky & William Rudman Abstract Understanding when Vision-Language Models (VLMs) will behave unexpectedly, whether models can reliably predict their own behavior, and if models adhere to their introspective reasoning are central challenges for trustworthy deployment. To study this, we introduce the Graded Color Attribution (GCA) dataset, a controlled benchmark designed to elicit decision rules and evaluate participant faithfulness to these rules. GCA consists of line drawings that vary pixel-level color coverage across three conditions: “world-knowledge” recolorings, counterfactual recolorings, and shapes with no color priors. Using GCA, both VLMs and human participants establish a threshold: the minimum percentage of pixels of a given color an object must have to receive that color label. We then compare these rules with their subsequent color attribution decisions. Our findings reveal that models systematically violate their own introspective rules. For example, GPT-5-mini violates its stated introspection rules in nearly 60% of cases on objects with strong color priors. Human participants remain faithful to their stated rules, with any apparent violations being explained by a well-documented tendency to overestimate color coverage. In contrast, we find that VLMs are excellent estimators of color coverage, yet blatantly contradict their own reasoning in their final responses. Across all models and strategies for eliciting introspective rules, world-knowledge priors systematically degrade faithfulness in ways that do not mirror human cognition. Our findings challenge the view that VLM reasoning failures are difficulty-driven and suggest that VLM introspective self-knowledge is miscalibrated, with direct implications for high-stakes deployment. Figure 1: Responses from GPT-5 on Graded Color Attribution. When images are re-colored with world-knowledge colors, linguistic priors often undermine model faithfulness to their stated thresholds for color attribution. Responses are truncated for brevity. 1 Introduction As VLMs are increasingly deployed in high-risk domains such as medicine and law, the ability to produce trustworthy, logically consistent reasoning traces has become critical ( 13 ; 24 ) . Chain-of-Thought (CoT) prompting is often proposed as an explainability proxy, assuming outputs reflect internal reasoning. Despite producing coherent CoT traces, models may ignore stated premises, contradict derived conclusions, or yield answers inconsistent with the logic they appear to follow . In some cases, when asked to predict their behavior hypothetically, models report one course of action but behave differently in practice ( 9 ) . We call this a failure to obey their own introspective rules , the internal constraints a model derives for itself and claims it will use in its responses. In this work, we elicit introspective rules through targeted CoT prompting strategies, compare these rules against human judgments, and examine cases in which models diverge from humans and contradict their stated rules. Prior approaches to evaluating CoT adherence rely on counterfactual perturbations or hint injection to measure whether model responses shift accordingly ( 18 ) . Other work has drawn parallels between model inconsistencies and human cognitive biases, arguing that task difficulty determines CoT faithfulness ( 3 ) . To that end, many studies argue that it is necessary to decouple CoT faithfulness from response correctness, as they are often uncorrelated ( 19 ; 16 ) . Our work addresses these gaps along several key dimensions. We introduce the Graded Color Attribution (GCA) dataset, which makes the base task trivial and decouples reasoning from correctness. The GCA dataset consists of two tasks: deriving a pixel threshold for color attribution and judging the color of partially recolored black-and-white images. These tasks are conceptually simple and allow for direct comparison to human introspective behavior. Although color label decisions are simple, they become subjective when white and recolored pixels are roughly equal in number. This avoids the pitfall noted by 21 , who argue that introspection is difficult to study when an obviously correct answer exists. Using GCA, we measure both the consistency of color-attribution rules for humans and VLMs and the faithfulness of decisions these stated rules. To test the impact of linguistic priors, GCA includes three stimulus types: (1) objects with strong color associations in aligned colors, (2) the same objects in counterfactual colors, and (3) shapes with no color priors. We find that both VLMs and humans produce largely consistent threshold rules; however, faithfulness to the stated rules degrades when presented with objects re-colored with aligned priors. Figure 1 shows (truncated) reasoning traces for GPT-5-mini on our GCA dataset. Here, the model derives a 50% threshold and notes that most apple pixels are white, yet still answers“red”. Human responses, by contrast, are invariant across stimulus types for both thresholds and reasoning faithfulness. Our findings contrast with accounts that frame CoT faithfulness failures as difficulty-driven, and against 3 who argue that CoT failures mirror human bias. Our contributions are as follows: 1. We propose the Graded Color Attribution (GCA) task, in which stylized black-and-white images are gradually re-colored across a range of pixel thresholds. GCA decouples both task difficulty and task correctness from faithfulness evaluations. 2. We examine VLM consistency across CoT strategies and recoloring thresholds. Introspective rules shift substantially with visual input, suggesting they are not stable internal representations. 3. We conduct human trials on GCA to assess alignment between humans and models on this targeted task and to demonstrate that model failures are not attributed to human cognitive biases. 4. We find that while models are consistent across different CoT variants, world-knowledge priors degrade faithfulness to their introspective rules. 1 1 1 Data: https://huggingface.co/datasets/mgolov/graded-color-attribution Code: https://github.com/wrudman/when_to_call_an_apple_red Figure 2: Examples from GCA showing prior-aligned (red ant), counterfactual (blue strawberry), and no-prior (yellow pentagon) recolorings. 2 Related Works A growing body of work questions whether chain-of-thought reasoning traces reflect the actual computations that produce model outputs ( 3 ; 1 ; 22 ; 11 ; 17 ; 20 ) . 9 shows that models can produce strategically misleading reasoning, while 5 demonstrates that alignment between stated reasoning and final answers degrades systematically as task difficulty increases. Most directly relevant, 19 introduce a discriminative benchmark for instance-level unfaithfulness that distinguishes two failure modes: post-hoc reasoning , where steps rationalize a predetermined answer, and spurious reasoning chains , where steps appear coherent but lack a causal link to the output. They find that both model scale and task domain shape faithfulness. Examining Cot faithfulness is more challenging in VLMs, where models must integrate visual and linguistic signals. 18 find that VLMs are less self-consistent than LLMs, with text dominating over image inputs during answer generation. 6 shows that intermediate reasoning steps are unreliable irrespective of final answer accuracy, while 16 identifies a “seeing but lying” phenomenon, where models achieve high perceptual scores but low faithfulness. Complementary work suggests this gap is not due to perception alone: models often attend to correct visual evidence yet fail to use it in reasoning ( 14 ) , rely primarily on text despite multimodal CoT ( 15 ) , and produce explanations that are not causally faithful under counterfactual interventions ( 7 ; 10 ) . Longer CoT can further exacerbate this, increasing hallucinations by shifting reliance toward language priors ( 12 ) . Several works examine how external cues shape model reasoning and whether models faithfully acknowledge those cues. 2

Read the complete text