Episode-specific Fine-tuning for Metric-based Few-shot Learners with Optimization-based Training

Xuanyu Zhuang, Geoffroy Peeters, Gaël Richard · 2025

Read Episode-specific Fine-tuning for Metric-based Few-shot Learners with Optimization-based Training by Xuanyu Zhuang, Geoffroy Peeters, Gaël Richard (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2506-17499.txt.

Introduction

Xuanyu Zhuang, Geoffroy Peeters, and Gael Richard wrote a 2025 paper on few shot learning with distance based models. In each small task, a few labeled samples must guide the sorting of new ones. Distance based models compare new samples to the labeled ones in a learned space, then assign the nearest label.

The labeled set, though, could also tune the space itself for the task at hand. The authors give three fine tuning tricks that build fake sample pairs from the labeled set, so even simple models can adapt per task. Scant data raise the risk of overfitting. They answer that risk by training the base model inside a meta learning frame that rewards fast task fit.

Catalog record

CREATORXuanyu Zhuang, Geoffroy Peeters, Gaël Richard
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE51 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2506-17499

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Episode-specific Fine-tuning for Metric-based Few-shot Learners with Optimization-based Training Xuanyu Zhuang, Geoffroy Peeters, Gaël Richard Abstract In few-shot classification tasks (so-called episodes), a small set of labeled support samples is provided during inference to aid the classification of unlabeled query samples. Metric-based models typically operate by computing similarities between query and support embeddings within a learned metric space, followed by nearest-neighbor classification. However, these labeled support samples are often underutilized—they are only used for similarity comparison, despite their potential to fine-tune and adapt the metric space itself to the classes in the current episode. To address this, we propose a series of simple yet effective episode-specific, during-inference fine-tuning methods for metric-based models, including Rotational Division Fine-Tuning (RDFT) and its two variants, Iterative Division Fine-Tuning (IDFT) and Augmented Division Fine-Tuning (ADFT). These methods construct pseudo support-query pairs from the given support set to enable fine-tuning even for non-parametric models. Nevertheless, the severely limited amount of data in each task poses a substantial risk of overfitting when applying such fine-tuning strategies. To mitigate this, we further propose to train the metric-based model within an optimization-based meta-learning framework. With the combined efforts of episode-specific fine-tuning and optimization-based meta-training, metric-based models are equipped with the ability to rapidly adapt to the limited support samples during inference while avoiding overfitting. We validate our approach on three audio datasets from diverse domains, namely ESC-50 (environmental sounds), Speech Commands V2 (spoken keywords), and Medley-solos-DB (musical instrument). Experimental results demonstrate that our approach consistently improves performance for all evaluated metric-based models (especially for attention-based models) and generalizes well across different audio domains. I Introduction By learning class-independent, transferable knowledge during training rather than class feature information, few-shot classification (FSC) models achieve generalization to unlabeled novel class samples (query set) with only few labeled examples available per class (support set) [ 1 ] . This allows FSC to overcome the limitations of scarce training data to some extent and enables effective learning for rare data. This capability has contributed to the growing popularity of few-shot classification (FSC) across a wide range of applications, particularly within audio-related research domains such as audio classification [ 2 , 3 , 4 , 5 , 6 ] , sound event detection [ 7 , 8 , 9 , 10 , 11 , 12 ] , and music information retrieval [ 13 , 14 , 15 , 16 ] . Specifically, metric-based models and optimization-based meta-learning approaches are among the most representative methods in FSC. Metric-based methods aim to learn an embedding space in which the similarity between query and support samples can be measured using a predefined distance function. Classification is then performed in a non-parametric manner, such as by applying a nearest-neighbor classifier [ 17 , 18 , 19 ] or directly computing a weighted sum over the labels of the support samples based on similarity [ 20 ] . During inference, since the embedding space—used as the comparison metric—has already been learned, the model can directly compare query and support samples from novel classes within this space to perform classification, where the labeled support samples effectively serve as cluster points. On the other hand, optimization-based FSC focuses on achieving efficient during-inference fine-tuning on the limited amount of labeled support set samples without overfitting. As a representative approach, Model-Agnostic Meta-Learning (MAML) [ 21 ] aim to learn a set of good initial model parameters that can quickly generalize to new tasks with only a few gradient updates, while its variants [ 22 , 23 ] additionally focus on discovering better parameterized optimization strategies during fine-tuning. In our prior work [ 24 ] , we investigated the feasibility of performing episode-specific fine-tuning during inference for metric-based few-shot classification models, such as the Prototypical Network (PN) [ 17 ] , analogous to the approach employed in optimization-based methods. Related works have attempted similar fine-tuning strategies but achieved only marginal improvements [ 25 , 26 ] . Others have explored combining metric-based models with optimization-based approaches, yet without incorporating fine-tuning within their scope [ 27 , 28 ] . A key challenge we identified is that fine-tuning metric-based models typically requires both a labeled support set and a labeled query set to compute the loss. However, obtaining labeled query samples during inference is impractical, which limits the applicability of conventional fine-tuning techniques in this context. To this end, we proposed Rotational Division Fine-Tuning (RDFT) to manually construct pairs of labeled pseudo query sets and pseudo support sets from the given support set for fine-tuning. Integrating RDFT, we then adopted PN as the target learner of optimization-based frameworks, in order to equip PN with the ability of fast adaption on few available support samples without overfitting. Such approach successfully achieved a clear improvement for PN on few-shot audio classification tasks with two datasets from different audio domains, namely ESC-50 (environmental sounds) [ 29 ] and Speech Commands V2 (spoken keywors) [ 30 ] . Inheriting from our previous research, this paper includes the following contributions: • We extend RDFT beyond Prototypical Networks (PN) by applying it to other representative metric-based models, including Matching Networks (MN) [ 20 ] and Cross Attention Networks (CAN) [ 18 ] , to demonstrate its effectiveness across diverse metric-based architectures. Meanwhile, a third music dataset Medley-solos-DB [ 31 ] is also introduced to further broaden the experimental evaluation. • We further propose two variants of RDFT, namely Iterative Division Fine-tuning (IDFT) and Augmented Division Fine-tuning (ADFT), which modify RDFT from two opposite directions. • Specifically, for ADFT, we further explore the incorporation of audio augmentation techniques into the fine-tuning process, and investigate their particular significance for attention-based models. The rest of the paper is organized as follows: In Section II , we discuss the working mechanisms of few-shot classification, as well as of the backbone structures that are used in this work. In Section III , we provide the details of our proposed episode-specific fine-tuning methods and frameworks for metric-based models. We then report the experimental settings and results, which validate our proposal, in Section IV . II Preliminaries II-A Few-shot Classification Few-shot Classification (FSC) aims to develop classifiers that can easily generalize to previously unseen classes using only a small number of labeled samples per class. FSC tasks are typically formalized as K -way- N -shot episodes, where a model must classify unlabeled query samples (query set) among K novel classes, each represented by N labeled samples (support set). To ensure the generalization capability of FSC models, the dataset is split by class rather than by sample. That is, the test set contains only samples from classes that are entirely disjoint from those used during training. Vinyals et al. first proposed the episodic training principle for few-shot learning in [ 20 ] , in which the core idea is to mimic the test scenario during training. Specifically, training batches are constructed as K -way- N -shot episodes by randomly selecting K classes and N samples per class from the training set. To maintain consistency, episodic training principle requires the same values of K and N being used for constructing episodes during both training and evaluation. This training paradigm has been widely adopted and proven effective across various FSC scenarios and

Read the complete text