MuRF: Unlocking the Multi-Scale Potential of Vision Foundation Models

Bocheng Zou † , Mu Cai † , Mark Stanley ∗ , Dingfu Lu ∗ , and Yong Jae Lee · 2026

Read MuRF: Unlocking the Multi-Scale Potential of Vision Foundation Models by Bocheng Zou † , Mu Cai † , Mark Stanley ∗ , Dingfu Lu ∗ , and Yong Jae Lee (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-25744.txt.

Introduction

Zou and co-authors present MuRF, a way to use vision foundation models at several image scales at once. Most systems run inference at one fixed resolution. Low resolution favors broad scene meaning, while high resolution captures fine detail. MuRF runs a frozen model on multiple sizes of the same image. It then fuses the features into one representation. The method needs no training and fits many model families. The authors test it mainly on DINOv2 across many tasks. They also show it transfers to SigLIP2.

Catalog record

CREATORBocheng Zou † , Mu Cai † , Mark Stanley ∗ , Dingfu Lu ∗ , and Yong Jae Lee
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE38403
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2603.25744

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

MuRF: Unlocking the Multi-Scale Potential of Vision Foundation Models Bocheng Zou † , Mu Cai † , Mark Stanley ∗ , Dingfu Lu ∗ , and Yong Jae Lee Abstract Vision Foundation Models (VFMs) have become the cornerstone of modern computer vision, offering robust representations across a wide array of tasks. While recent advances allow these models to handle varying input sizes during training, inference typically remains restricted to a single, fixed scale. This prevalent single-scale paradigm overlooks a fundamental property of visual perception: varying resolutions offer complementary inductive biases, where low-resolution views excel at global semantic recognition and high-resolution views are essential for fine-grained refinement. In this work, we propose Mu lti- R esolution F usion ( MuRF ), a simple yet universally effective strategy to harness this synergy at inference time. Instead of relying on a single view, MuRF constructs a unified representation by processing an image at multiple resolutions through a frozen VFM and fusing the resulting features. The universality of MuRF is its most compelling attribute. It is not tied to a specific architecture, serving instead as a fundamental, training-free enhancement to visual representation. We empirically validate this by applying MuRF to a broad spectrum of critical computer vision tasks across multiple distinct VFM families—primarily DINOv2, but also demonstrating successful generalization to contrastive models like SigLIP2. = Date : March 26, 2026 = Projects : https://MuRF-VFM.github.io Code Repository : https://github.com/orgs/MuRF-VFM = Contact : [email protected] 1 Introduction The landscape of computer vision has been reshaped by standardizing visual representation learning through large-scale Vision Foundation Models (VFMs) ( 22 ) . These models, pre-trained on vast datasets, provide widely transferable features that serve as the bedrock for numerous downstream applications. A key axis of evolution in this domain has been handling image resolution. While early ViT-based approaches required rigid, fixed-size inputs ( 7 ) , recent paradigms like “multi resolution” training (e.g., DINOv2 ( 22 ) ) have enabled models to process images of arbitrary aspect ratios and scales flexibly. Figure 1 : The “Recognition vs. Refinement” Dynamic. The feature map obtained when the input is resized to 266, 518, 784. At lower resolutions, the representation is globally coherent, enabling robust recognition . At higher resolutions, boundary details are sharper, enabling precise refinement , but the object’s interior becomes noisy, risking incomplete segmentation. Our work is motivated by synergizing these two roles. Despite this flexibility in training , standard inference protocols remain surprisingly rigid. Typically, an input image is resized to a single “optimal” resolution before being processed by the VFM. We argue that this single-scale inference discards a wealth of information inherent in the multi-scale nature of visual data. As shown in Figure 1 , it is well-understood that a “division of labor” exists across different viewing scales: low-resolution inputs, by virtue of their larger relative patch sizes, excel at capturing globally coherent semantic context (recognition). High-resolution inputs are indispensable for resolving fine-grained high-frequency details and precise boundaries (refinement). By restricting inference to any single scale, current methods inevitably compromise either global coherence or local precision. We posit that unlocking the full potential of pre-trained VFMs requires actively aggregating these complementary views at inference time. To this end, we introduce Mu lti- R esolution F usion ( MuRF ), a general and straightforward strategy designed to create a scale-robust visual representation. Rather than modifying the VFM backbone, MuRF treats it as a frozen feature extractor. It processes the same input image across a pyramid of resolutions and fuses the resulting features into a unified representation. This simple operation effectively synergizes the recognition strengths of low-resolution views with the refinement capabilities of high-resolution views, without requiring expensive multi-scale training of the backbone itself. The universality of MuRF is its most compelling attribute. It is not tied to a specific architecture or task but serves as a fundamental enhancement to the visual representation. We empirically validate this by applying MuRF to a broad spectrum of critical computer vision tasks, all using frozen VFM backbones: • Dense Prediction (Probing): In semantic segmentation and depth estimation, simple linear heads trained on MuRF representations significantly outperform those trained on standard single-scale features, proving that multi-scale fusion enriches basic feature quality. • Multimodal Understanding: When used as the visual encoder for Multimodal Large Language Models (MLLMs) in tasks like Visual Question Answering (VQA), MuRF provides LLMs with a more comprehensive visual context, enabling them to reason about both macroscopic scenes and microscopic details simultaneously. • Unsupervised Anomaly Detection: In this training-free regime, MuRF effectively resolves the trade-off between detecting large structural anomalies and tiny surface defects, achieving state-of-the-art performance on demanding benchmarks like MVTec AD 2 TEST priv,mix ( 13 ) . Our results suggest that effectively leveraging multi-resolution views at inference time is a general principle that consistently improves VFM-based systems. MuRF offers a simple, unified, and highly effective implementation of this principle. Figure 2 : Overview of Mu lti- R esolution F usion ( MuRF ). An input image is resized to multiple resolutions and each view is processed by a frozen DINOv2 encoder to produce separate feature maps. These features are upsampled to a shared spatial resolution and fused into a single multi-resolution representation, which can then be used by lightweight task-specific heads for semantic segmentation, depth estimation, visual question answering, visual grounding, and other downstream tasks. Background is removed from the PCA figures. 2 Related Work Our work explores unlocking the full potential of frozen VFMs by revisiting and modernizing multi-resolution inference strategies. We review the evolution of VFMs regarding standard input resolutions and discuss historical and contemporary approaches to multi-scale representation learning. 2.1 Vision Foundation Models and Input Resolution The field of computer vision has been fundamentally altered by the rise of Vision Foundation Models (VFMs). Initially dominated by Convolutional Neural Networks (CNNs) like ResNet ( 12 ) trained via supervised learning, the paradigm shifted toward Vision Transformers (ViTs) ( 7 ) leveraging large-scale self-supervision. Models such as CLIP ( 23 ) and DINO/DINOv2 ( 3 , 22 ) now serve as ubiquitous feature extractors across diverse tasks. Traditionally, ViTs have been constrained by rigid input resolutions (e.g., 224\times 224 ) due to their reliance on fixed positional embeddings. This limitation often necessitates aggressive resizing or cropping, potentially discarding critical visual information. Recent advancements, such as DINOv2 ( 22 ) , NaViT ( 6 ) and FlexViT ( 2 ) , have introduced multi resolution or native resolution training, allowing models to process images in varying sizes during training. While multi resolution training enhances model flexibility, standard inference protocols typically revert to utilizing a single, fixed scale, often the highest resolution practically affordable. This single-scale inference overlooks the inherent “division of labor" in visual perception: low-resolution views excel at global semantic recognition, while high-resolution views are necessary for fine-grained refinement. Our Mu lti- R esolution F usion ( MuRF ) strategy addresses this by explicitly aggregating features from multiple resolutions at inference time, ensuring that both global context and local details are preserved without retraining the backbone. 2.2 Multi-Scale Representations Handling objects of varying scales is a classical challenge in computer vision. Early approaches relied on image pyramids ( 1 ) , where an image is repeatedly resized

Read the complete text