Contrasting Cognitive Styles in Vision-Language Models: Holistic Attention in Japanese Versus Analytical Focus in English

Ahmed Sabir Azinovič Gasper Mengsay Loem Rajesh Sharma · 2025

Read Contrasting Cognitive Styles in Vision-Language Models: Holistic Attention in Japanese Versus Analytical Focus in English by Ahmed Sabir Azinovič Gasper Mengsay Loem Rajesh Sharma (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2507-00700.txt.

Introduction

Ahmed Sabir and co-authors test whether vision language models inherit the seeing style of their training language. Their paper from 2025 draws on cross culture studies: East Asian viewers tend to take in a whole scene and its links, while Western viewers lock onto single objects and traits. Japanese and English trained models are compared.

Image descriptions from the two model groups were set side by side for wide versus narrow focus. The Japanese trained models leaned toward scene level, link rich accounts, while the English trained ones leaned to object level detail. The team reads this as proof that training data carries cultural cognition into model output, past grammar alone.

Catalog record

CREATORAhmed Sabir Azinovič Gasper Mengsay Loem Rajesh Sharma
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE33 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2507-00700

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Contrasting Cognitive Styles in Vision-Language Models: Holistic Attention in Japanese Versus Analytical Focus in English Ahmed Sabir Azinovič Gasper Mengsay Loem Rajesh Sharma Abstract Cross-cultural research in perception and cognition has shown that individuals from different cultural backgrounds process visual information in distinct ways. East Asians, for example, tend to adopt a holistic perspective, attending to contextual relationships, whereas Westerners often employ an analytical approach, focusing on individual objects and their attributes. In this study, we investigate whether Vision-Language Models (VLMs) trained predominantly on different languages, specifically Japanese and English, exhibit similar culturally grounded attentional patterns. Using comparative analysis of image descriptions, we examine whether these models reflect differences in holistic versus analytic tendencies. Our findings suggest that VLMs not only internalize the structural properties of language but also reproduce cultural behaviors embedded in the training data, indicating that cultural cognition may implicitly shape model outputs. 1 Introduction Figure 1: Overview of our analysis framework. Our methodology identifies whether image descriptions exhibit holistic or analytic tendencies. First, we filter and select images using four visual classifiers combined with object similarity voting. Next, we generate image captions using VLMs, e.g . Gemma-3 for English and VILA-JP for Japanese. Finally, we use GPT-4o with few-shot in-context learning to classify each caption as holistic or analytic. The influence of culture on human cognition is profound, shaping distinct patterns of thought and perception across societies Masuda and Nisbett 2001 ; Masuda and Nisbett 2006 . For example, individuals from East Asian and North American backgrounds differ fundamentally in how they interpret visual information. These divergences are often linked to intellectual traditions: analytic thinking rooted in ancient Greek philosophy emphasizes object-focused reasoning, while East Asian traditions favor holistic thinking, highlighting the interplay between objects and their contexts Masuda 2017 . These culturally shaped visual processing strategies extend to digital media and information foraging behaviors. Studies on mock webpages show that East Asian participants are more efficient at processing large volumes of information compared to North Americans Wang et al. 2012 . Baughan et al. 2021 found that American users tend to rely on salient cues for rapid searching, whereas Japanese users scan content more broadly, prioritizing completeness over speed. In visual storytelling, such differences are also evident. A cross-cultural analysis of comics revealed that Japanese manga often employs minimalist or blank backgrounds, assuming contextual inference by the reader, while American and European comics depict detailed settings more explicitly Atilla et al. 2023 . These stylistic choices reflect broader cognitive expectations about the reader’s attention and interpretation. Cultural differences in visual interpretation are also mirrored in linguistic patterns. Multilingual image descriptions, such as those in Japanese and Korean, tend to encode a broader range of semantic attributes and relationships than monolingual ( e.g . English) descriptions Ye et al. 2024 . Together, these findings from psychology, perception, and human-computer suggest that culturally distinct modes of visual processing are internalized and persist over time. This raises a compelling question for the development of artificial intelligence. Given these human-like cultural divergences, do advanced systems such as Vision-Language Models (VLMs) also exhibit culturally influenced patterns of attention and description? Specifically, can models trained or fine-tuned on language-specific data, such as Japanese versus English, demonstrate holistic versus analytic tendencies akin to those seen in human observers from corresponding cultures? This study explores whether VLMs inherit culturally situated patterns through their training data. We focus on comparing the descriptive outputs of models exposed primarily to Japanese versus English, analyzing the extent to which their outputs reflect holistic (context-sensitive) versus analytic (object-focused) styles. Our findings suggest that these models not only absorb linguistic structures but also encode cultural biases in visual interpretation, echoing attentional and descriptive tendencies observed in human cognition. 2 Problem Statement and Definitions We investigate whether VLMs trained in different languages reflect culturally influenced cognitive styles, specifically, whether Western models produce more analytic image descriptions, while Eastern models favor holistic descriptions. We define: Object as the primary subject in an image e.g . cat or a bird are considered objects; Background as contextual elements excluding the main object e.g . sky or a dining room are considered Background; Analytic description as text centered on objects; and Holistic description as text emphasizing the broader scene. 3 Dataset We use the COCO Caption dataset Chen et al. 2015 , which contains 120K images with five human-written captions per image. It includes 82,783 training and 40,504 validation images. For our experiments, we randomly select 7K images from the validation set. A filtering process is applied to retain only images with confidently detectable objects, resulting in a subset of 5K images. In addition, to explore cultural differences in image description more robustly, we also evaluate on: (1) HERON Benchmark Inoue et al. 2024 : 21 culturally-relevant Japanese images across 7 categories (anime, art, culture, food, landscape, landmark, transportation), (2) JA-VLM Akiba et al. 2024 : 42 Japanese cultural images from the wild, (3) 1K Japanese Image Dataset (ours): 6435 Japanese photos filtered to 1K using human validation. 1 1 1 https://huggingface.co/datasets/Japanese-photos 1K % 2K % 5K % Model Method Holistic Analytic Holistic Analytic Holistic Analytic GPT4o OpenAI 2024 5-shot 22.60 77.40 20.25 79.75 19.22 80.78 GPT4o (balanced) 6-shot 20.80 79.20 21.05 78.95 19.80 80.20 GPT4o-JP 5-shot 24.60 75.40 24.15 75.85 23.64 76.36 GPT4o-JP (balanced) 6-shot 26.10 73.90 27.75 72.25 28.64 71.36 Gemma-3-12B Team et al. 2025 5-shot 13.40 86.60 13.30 86.70 14.16 85.84 Gemma-3-12B (balanced) 6-shot 13.70 86.30 13.15 86.85 13.78 86.22 VILA-JP-14B Sasagawa et al. 2025 5-shot 11.60 88.40 12.80 87.20 13.88 86.12 VILA-JP-14B (balanced) 6-shot 14.70 85.30 15.75 84.25 16.90 83.10 Gemma-3-27B-JP 5-shot 13.00 87.00 13.00 87.00 13.50 86.50 Gemma-3-27B-JP (balanced) 6-shot 19.70 80.30 19.55 80.45 21.14 78.86 Table 1: Comparison analysis results of different English and Japanese Vision-LLMs on a subset of the COCO dataset. GPT-4o serves as the evaluator, judging whether each model’s image caption begins with a background-focused ( Holistic ) or object-focused ( Analytic ) description. The results indicate that larger models, such as GPT-4o, better replicate cultural cognitive styles. Bold indicates the highest score among large models, while Underline indicates the highest score among smaller models. HERON-BENCH % JA-VLM % 1K- JP-dataset % Model Method Holistic Analytic Holistic Analytic Holistic Analytic GPT4o 5-shot 25.58 74.42 23.81 76.19 59.10 40.90 GPT4o (balanced) 6-shot 27.91 72.09 19.05 80.95 57.60 42.40 GPT4o-JP 5-shot 34.88 65.12 38.10 61.90 63.86 36.14 GPT4o-JP (balanced) 6-shot 53.49 46.51 61.90 38.10 73.37 26.63 Gemma-3-12B 5-shot 34.88 65.12 19.05 80.95 54.10 45.90 Gemma-3-12B (balanced) 6-shot 27.91 72.09 14.29 85.71 54.80 45.20 VILA-JP-14B 5-shot 30.23 69.77 28.57 71.43 55.10 44.90 VILA-JP-14B (balanced) 6-shot 41.86 58.14 33.33 66.67 66.00 34.00 Gemma-3-27B-JP 5-shot 27.91 72.09 28.57 71.43 65.20 34.80 Gemma-3-27B-JP (balanced) 6-shot 39.53 60.47 33.33 66.67 64.60 35.40 Table 2: Comparison analysis results examine different English and Japanese Vision-LLMs on Japanese cultural image benchmark datasets, including HERON-BENCH, JA-VLM, and our 1K JP-dataset. The results show that the Holistic ratio increases in Japanese models and the Analytic decreases in English models, when compared to the Western COCO dataset, suggesting that the dataset also contributes to certain cognitive styles. 4 Methodology Our framework consists of three main stages: image selection, caption generation, and textual classification via in-context learning (see Figure

Read the complete text