HyperFlow: Gradient-Free Emulation of Few-Shot Fine-Tuning

Donggyun Kim, Chanwoo Kim, Seunghoon Hong · 2025

Read HyperFlow: Gradient-Free Emulation of Few-Shot Fine-Tuning by Donggyun Kim, Chanwoo Kim, Seunghoon Hong (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2504-15323.txt.

Introduction

Donggyun Kim, Chanwoo Kim, and Seunghoon Hong present HyperFlow in 2025, a method that mimics fine-tuning without computing gradients. Fine-tuning at test time helps few-shot learning, where only a few examples are given, but repeated gradient steps cost too much for real-time or low-resource use.

They treat gradient descent as steps along a differential equation, then train a helper network to predict that drift from the few examples alone. Adapting a model then needs only a few forward passes of the helper, with no gradients and no passes through the target model. On two cross-domain few-shot benchmarks, the method raised out-of-domain scores well above the plain baseline. It used only a small share of the memory and time of standard fine-tuning.

Catalog record

CREATORDonggyun Kim, Chanwoo Kim, Seunghoon Hong
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE39 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2504-15323

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

HyperFlow: Gradient-Free Emulation of Few-Shot Fine-Tuning Donggyun Kim, Chanwoo Kim, Seunghoon Hong Abstract While test-time fine-tuning is beneficial in few-shot learning, the need for multiple backpropagation steps can be prohibitively expensive in real-time or low-resource scenarios. To address this limitation, we propose an approach that emulates gradient descent without computing gradients, enabling efficient test-time adaptation. Specifically, we formulate gradient descent as an Euler discretization of an ordinary differential equation (ODE) and train an auxiliary network to predict the task-conditional drift using only the few-shot support set. The adaptation then reduces to a simple numerical integration ( e.g., via the Euler method), which requires only a few forward passes of the auxiliary network—no gradients or forward passes of the target model are needed. In experiments on cross-domain few-shot classification using the Meta-Dataset and CDFSL benchmarks, our method significantly improves out-of-domain performance over the non-fine-tuned baseline while incurring only 6 % 6\% of the memory cost and 0.02 % 0.02\% of the computation time of standard fine-tuning, thus establishing a practical middle ground between direct transfer and fully fine-tuned approaches. 1 Introduction Few-shot classification aims to recognize novel classes of images using only a handful of labeled examples per class [ 15 , 31 , 51 , 41 ] . Unlike conventional supervised learning approaches that use an extensive amount of labeled images for pre-defined tasks to train specialized models, few-shot learning approaches focus on adapting a pretrained model to novel tasks using a few examples, where the target classes and image domains can vary at test time. This paradigm is vital for real-world scenarios where the image domain is distinctive and the labeled data is scarce or expensive to obtain, such as medical imaging [ 30 ] , plant disease classification [ 2 ] , and remote sensing [ 48 ] . Even in the era of increasing general-purpose foundation models [ 4 ] , few-shot learning is still beneficial to achieving optimal performance in domain-specific contexts [ 34 , 52 , 36 ] . Recent advances in few-shot classification often rely on an additional fine-tuning stage using the few examples given at test time. Thanks to the simplicity and effectiveness of their design, embedding-based algorithms [ 51 , 47 , 3 ] have been predominant in few-shot classification tasks. These methods classify images by learning a shared embedding space and assigning labels based on distances or similarities to reference points ( e.g., class prototypes or exemplars) within that space. While these approaches were originally designed to directly transfer a pretrained feature extractor to novel classes without adaptation, recent studies have consistently shown that additional fine-tuning at test time significantly improves performance, particularly when substantial domain discrepancies exist between training and testing scenarios [ 17 , 40 , 24 , 38 , 23 ] . However, despite its effectiveness, fine-tuning incurs substantial computational costs, both in memory and time, making it impractical for many real-world few-shot learning scenarios such as resource-constrained environments or real-time applications. This high computational overhead primarily results from the repeated backpropagation required during gradient descent. Therefore, achieving a balance between direct transfer and fine-tuning necessitates a computationally efficient adaptation mechanism that avoids gradient-based optimization entirely. This perspective shares a high-level inspiration with parameter-generation methods [ 18 , 29 , 39 , 45 , 46 ] , which employ hypernetworks to generate model parameters. However, most existing methods primarily focus on modeling parameter distributions for a pre-defined task with different architectures or training configurations, and relatively little attention has been paid to adapting pretrained parameters for novel classification tasks given only a few examples. In this paper, we propose HyperFlow , a model-agnostic adaptation mechanism for few-shot classification that emulates fine-tuning without computing any gradients. To achieve this, we introduce an auxiliary drift network that learns the parameter dynamics of the ordinary differential equations (ODEs) induced by the gradient descent ( i.e., gradient flows). As the gradient flow depends on the tasks, we amortize them by conditioning the drift network on a few support examples. Since directly learning the ODEs in the full parameter space of modern neural networks is infeasible, we adopt a parameter-efficient fine-tuning (PEFT) approach [ 53 ] , which reduces the total parameters that need to be updated and thereby enables scalable training and inference. In addition, we approximate the smooth gradient flows by interpolating the discrete gradient descent trajectories collected from an offline simulation on the meta-training dataset, enabling efficient training of the conditional drift network. As a result, the fine-tuning stage can be replaced with a lightweight ODE-solver ( e.g., Euler method) that involves only a few forward passes through the drift network. The drift network can be significantly smaller and more computationally efficient than the target models, thus incurring negligible computational overhead compared to gradient-based fine-tuning methods. We demonstrate our approach in cross-domain few-shot classification settings using widely adopted benchmarks such as Meta-Dataset [ 49 ] and CDFSL [ 17 ] . When applied to the state-of-the-art few-shot classification approach [ 23 ] , HyperFlow can significantly improve the out-of-domain generalization by only using 6\% of the memory cost and 0.02\% of the computation time compared to conventional fine-tuning. As a result, HyperFlow offers a range of trade-offs between the computation cost and the performance and thus suggests the practical middle-ground between the direct transfer and fine-tuning approaches. 2 Preliminary Problem Setup Few-shot classification (FSC) aims to recognize novel classes of images using only a few labeled examples per class. To achieve this, a model is first trained on a base dataset \mathcal{D}_{\text{base}}=\{(x_{i},y_{i})\}_{i=1}^{N_{\text{base}}} , often called as “meta-training” dataset, which consists of many labeled examples for known base classes \mathcal{C}_{\text{base}} . During testing, the model is evaluated on multiple classification tasks involving novel classes \mathcal{C}_{\text{novel}} , where \mathcal{C}_{\text{base}}\cap\mathcal{C}_{\text{novel}}=\phi . To learn the novel classes of each test task \mathcal{T} , the model receives a small set of labeled examples \mathcal{S}_{\mathcal{T}}=\{(x_{j},y_{j})\}_{j=1}^{N_{\mathcal{T}}} called the support set . Then, the model is tested to classify unlabeled images within the same task, called the query set . In conventional N -way K -shot setting [ 51 , 16 ] , each task consists of N classes, and K labeled images per class are given as the support set. Additionally, we consider the more challenging cross-domain few-shot classification problem, where the testing images originate from domains different from those of the base dataset \mathcal{D}_{\text{base}} ( e.g., domain shifts from natural images to X-ray or satellite imagery). In this setup, the model must equip strong generalization ability and flexible adaptation mechanism to address both novel classes and out-of-domain images. P > M > F Pipeline for FSC While various approaches have been proposed to tackle the FSC problem, a common training pipeline involves one or more of three distinct stages—Pre-training, Meta-training, and Fine-tuning—collectively known as P > M > F [ 23 ] . In the pre-training stage, general-purpose representations are learned using backbones trained with self-supervised methods on large-scale datasets ( e.g., the DINO backbone [ 6 ] pre-trained on ImageNet [ 11 ] ). During the meta-training stage, the model acquires prior knowledge about the underlying task distribution, typically by simulating few-shot learning episodes using the base dataset \mathcal{D}_{\text{base}} [ 51 ] . Finally, the

Read the complete text