LLMind : Bio-inspired Training-free Adaptive Visual Representations for Vision-Language Models
Soumyaratna DebnathBui Duc ManhZinan Liu, Lin WangNanyang Technological University, Singapore [email protected] [email protected] [email protected] [email protected] · 2026
Read LLMind : Bio-inspired Training-free Adaptive Visual Representations for Vision-Language Models by Soumyaratna DebnathBui Duc ManhZinan Liu, Lin WangNanyang Technological University, Singapore [email protected] [email protected] [email protected] [email protected] (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-14882.txt.
Introduction
Debnath and co-authors give vision-language models a human-like uneven gaze with LLMind, a training-free frame. Standard models spend equal precision on dull and rich image zones. Human vision instead magnifies the center and adapts on the fly. A bio-inspired sampling strategy warps pixels under tight budgets. A Möbius-shaped module keeps global scene structure while sampling unevenly. Closed-loop semantic feedback then aligns focus with the text query. Gains average +20% on VQAv2 and near +38% on Seed-Bench, they report.
Catalog record
| CREATOR | Soumyaratna DebnathBui Duc ManhZinan Liu, Lin WangNanyang Technological University, Singapore [email protected] [email protected] [email protected] [email protected] |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 38596 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2603.14882
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
LLMind : Bio-inspired Training-free Adaptive Visual Representations for Vision-Language Models Soumyaratna DebnathBui Duc ManhZinan Liu, Lin WangNanyang Technological University, Singapore [email protected] [email protected] [email protected] [email protected] Abstract Vision-Language Models (VLMs) typically assume a uniform spatial fidelity across the entire field of view of visual inputs, dedicating equal precision to even the uninformative regions. By contrast, human vision is neither uniform nor static. It is adaptive, selective, and resource-efficient. In light of this, we present the first systematic analysis of bio-inspired visual representation methods, providing insights for more efficient and adaptive VLMs. We propose LLMind ( Looking Like the Mind ), a novel training-free framework that mimics foveated encoding and cortical magnification in human vision to achieve adaptive , efficient representations for VLMs under tight pixel budgets. Our key idea is to explore a Bio-inspired Adaptive Sampling Strategy ( BASS ). This empowers us to design a Möbius-parameterized module that performs non-uniform sampling while preserving global scene structure. On top of BASS, we introduce closed-loop semantic feedback ( CSF ) via test-time adaptation to align the perceptual saliency with the textual information from the frozen VLM. We evaluate LLMind against uniform and other sampling baselines across diverse scene-level and region-guided visual question answering (VQA) benchmarks. The results show that ours achieves dramatic gains, with average improvements by +20 % on VQAv2, +38 % on Seed-Bench, and +37 % on A-OKVQA compared to uniform sampling under tight pixel budgets. More surprisingly, results reveal that LLMind can retain up to 82 %, 92 % and 97 % of the full-resolution performance with only 1%, 3% and 5% of the pixels, respectively. Moreover, LLMind is lightweight, plug-and-play, and compatible with existing VLMs without requiring architectural changes. Figure 1 : (a) Illustration of the underlying principle of our Bio-inspired Adaptive Sampling Strategy (BASS). (b) Performance comparison at 5% and 3% pixel budgets using Qwen2.5-VL Bai et al. 2025 across datasets. Project page: https://empactlab.github.io/LLMind-CVPR-2026/ $\dagger$ $\dagger$ footnotetext: Corresponding author. 1 Introduction Vision-Language Models (VLMs) have recently demonstrated impressive progress in multimodal reasoning and visual question answering (VQA) Li et al. 2023b ; Wang et al. 2024 ; Alayrac et al. 2022 ; Dai et al. 2023 . By jointly encoding visual and textual cues, these models can infer semantic correspondences and generate contextually coherent responses. However, despite these advances, existing models still operate under an assumption: perception requires uniform access to every pixel of an image Wang et al. 2025 ; Schwinn et al. 2022 . For instance, typical VLMs, e.g . , Qwen Yang et al. 2025 or LLaVA Li et al. 2024 based on attention mechanisms do not employ attention to modulate the spatial sampling of visual inputs Khan et al. 2022 . Attention in transformers redistributes representational emphasis across features, but the underlying image is still uniformly sampled at fixed resolution. Consequently, VLMs allocate equal computational resources to semantically irrelevant background regions and critical foreground details alike Han et al. 2022 ; Brauwers and Frasincar 2021 . While computationally tractable at small scales, these models become inefficient as image resolution increases, forcing models to uniformly downsample and, consequently, discard critical details from all regions of the image. Although dynamic tokenization Bolya et al. 2022 ; Rao et al. 2021 has recently been introduced to address such issues, it still requires full-resolution input, which limits its suitability for edge systems. From a neuroscience perspective, uniform downsampling does not reflect how biological vision allocates computational resources Wang et al. 2025 ; Schwinn et al. 2022 . Instead, the human visual system employs a hierarchical, attention-driven foveation strategy: the fovea provides high-acuity detail in a small attended region, while the peripheral retina conveys coarse, context-rich information that guides subsequent gaze shifts and processing Strasburger et al. 2011 (Fig. 1 (a)). Through rapid saccadic movements, the visual system dynamically repositions this high-resolution window, sampling the scene adaptively based on task demands Hayhoe and Ballard 2005 ; Land and Tatler 2009 . This mechanism reflects the principle of cortical magnification , where perceptually salient regions occupy disproportionately large representational space in the visual cortex Duncan 1984 . Perception, therefore, is not a static snapshot but an active optimization process that maximizes information gain while minimizing computational expenditure Friston 2010 . Inspired by the foveated encoding and cortical magnification in human vision, an interesting question arises: Can biologically inspired sampling strategies enable VLMs to achieve higher reasoning efficiency and accuracy than conventional uniform sampling under limited pixel budgets? To answer this, we present the first comprehensive analysis of the visual representation strategies in VLMs, evaluating their effectiveness against uniform sampling across multiple VQA benchmarks. Building on this, we propose LLMind , a novel adaptive sampling framework that emulates the human visual system for the visual stimuli. 1 1 1 Here, visual stimuli correspond to the input image of VLMs. Our core idea is to explore a Bio-inspired Adaptive Sampling Strategy ( BASS ) (Sec. 3.1 ). BASS serves as a computational analogue of cortical magnification, wherein the foveal region occupies a disproportionately large cortical representation Daniel and Whitteridge 1961 . To make it possible, we use Möbius transformation Arnold and Rogness 2008 ; Olsen 2010 , a mathematical tool that is parameterized to simulate cortical magnification. As a conformal mapping on the spherical plane, Möbius transformations perform smooth mappings of the image space ( i.e . rotation, translation, scaling, inversion) while preserving local geometry. As such, we can dynamically remap the spatial domain of an image, magnifying task-relevant regions with higher sampling density while compressing peripheral areas. We optimize the BASS parameters in a training-free loop using a perceptual loss, guided by the semantic output of the frozen VLM. Since the frozen VLM model is non-differentiable, we then introduce closed-loop semantic feedback ( CSF ) via test-time adaptation using Simultaneous Perturbation Stochastic Approximation Spall 2002 to estimate gradients directly from input-output interactions (Sec. 3.2 ). Unlike prior training-free methods that require access to internal model parameters or gradients of open ( white-box ) VLM Wu et al. 2024 , our gradient-free CSF strategy can also seamlessly integrate with proprietary ( black-box ) APIs, requiring no architectural modifications or parameter access. We extensively evaluate LLMind against uniform and other sampling baselines across diverse benchmark datasets, considering two complementary VQA paradigms: Scene-level VQA for holistic reasoning and Region-guided VQA for localized understanding. LLMind improves the average performance 2 2 2 Average across 1%, 3% and 5% pixel budgets. by +20% on VQAv2 Goyal et al. 2017 , +38% on Seed-Bench Li et al. 2023a , and +37% on A-OKVQA Schwenk et al. 2022 compared to uniform sampling under the same pixel budget for scene-level VQA tasks. For region-guided VQA, LLMind achieves an average gain of 35% over uniform sampling, even surpassing the full-resolution accuracy while using only 1% of the pixels (See Sec. 4.3 ). To summarise, the major contributions of this paper are: • Inspired by neuroscience-grounded knowledge, we identify a fundamental problem in VLMs and conduct the first comprehensive analysis of the visual representation strategies in VLMs. • We propose BASS that dynamically reallocates pixel budget to perceptually and semantically salient regions as a computational analogue of human vision (Sec. 3.1 ). • We introduce the CSF mechanism that aligns visual