Adaptive Clipping for Privacy-Preserving Few-Shot Learning: Enhancing Generalization with Limited Data
Kanishka Ranaweera, Dinh C. Nguyen, Pubudu N. Pathirana, David Smith, Ming Ding, Thierry Rakotoarivelo, Aruna Seneviratne · 2025
Read Adaptive Clipping for Privacy-Preserving Few-Shot Learning: Enhancing Generalization with Limited Data by Kanishka Ranaweera, Dinh C. Nguyen, Pubudu N. Pathirana, David Smith, Ming Ding, Thierry Rakotoarivelo, Aruna Seneviratne (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2503-22749.txt.
Introduction
Kanishka Ranaweera and co-authors add adaptive clipping to private few-shot learning to protect its accuracy. Their 2025 paper works where two scarcities meet: few labeled examples, and a formal privacy guarantee that usually costs model quality. Clipping, which caps how much any one example can sway training, is where much of that cost is decided.
Their method adjusts the clipping threshold as training proceeds instead of fixing it once. It fits several private meta-learning algorithms that learn how to learn from small data. Dynamic control limits how much sensitive information leaks while curbing overfitting on tiny sets. Tests show better generalization under the same privacy promise. Privacy tools that forget less could serve medicine and other fields where data are both scarce and sensitive.
Catalog record
| CREATOR | Kanishka Ranaweera, Dinh C. Nguyen, Pubudu N. Pathirana, David Smith, Ming Ding, Thierry Rakotoarivelo, Aruna Seneviratne |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 87 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2503-22749
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Adaptive Clipping for Privacy-Preserving Few-Shot Learning: Enhancing Generalization with Limited Data Kanishka Ranaweera, Dinh C. Nguyen, Pubudu N. Pathirana, David Smith, Ming Ding, Thierry Rakotoarivelo, Aruna Seneviratne Abstract In the era of data-driven machine-learning applications, privacy concerns and the scarcity of labeled data have become paramount challenges. These challenges are particularly pronounced in the domain of few-shot learning, where the ability to learn from limited labeled data is crucial. Privacy-preserving few-shot learning algorithms have emerged as a promising solution to address such pronounced challenges. However, it is well-known that privacy-preserving techniques often lead to a drop in utility due to the fundamental trade-off between data privacy and model performance. To enhance the utility of privacy-preserving few-shot learning methods, we introduce a novel approach called Meta-Clip. This technique is specifically designed for meta-learning algorithms, including Differentially Private (DP) model-agnostic meta-learning, DP-Reptile, and DP-MetaSGD algorithms, with the objective of balancing data privacy preservation with learning capacity maximization. By dynamically adjusting clipping thresholds during the training process, our Adaptive Clipping method provides fine-grained control over the disclosure of sensitive information, mitigating overfitting on small datasets and significantly improving the generalization performance of meta-learning models. Through comprehensive experiments on diverse benchmark datasets, we demonstrate the effectiveness of our approach in minimizing utility degradation, showcasing a superior privacy-utility trade-off compared to existing privacy-preserving techniques. The adoption of Adaptive Clipping represents a substantial step forward in the field of privacy-preserving few-shot learning, empowering the development of secure and accurate models for real-world applications, especially in scenarios where there are limited data availability. I Introduction Few-shot learning has emerged as a challenging and important problem in the field of machine learning [ 1 ] . Traditional learning algorithms typically require a large amount of labeled data to achieve satisfactory performance. However, in many real-world scenarios, obtaining such extensive labeled data is often impractical or prohibitively expensive. Few-shot learning aims to address this limitation by enabling models to learn from a small number of labeled examples, mimicking the human ability to generalize knowledge from limited experience. In the context of few-shot learning, meta-learning has gained considerable attention due to its ability to rapidly adapt to new tasks with limited data. Meta-learning algorithms leverage prior knowledge acquired from a distribution of related tasks to learn a meta-model capable of quickly adapting to new tasks. These algorithms hold promise for solving the few-shot learning problem by effectively utilizing shared information across tasks [ 2 , 3 ] . While meta-learning algorithms have shown promising results, it is essential to acknowledge the pressing privacy concerns that have been highlighted in machine learning research. Several works have documented privacy violations and risks associated with handling sensitive data in the context of machine learning models [ 4 , 5 , 6 ] . These issues encompass various attacks, data inference vulnerabilities, and membership inference threats that can compromise user privacy. It is crucial to recognize that these privacy challenges faced in machine learning are also relevant to the domain of meta-learning, given the potential for handling sensitive information in the meta-learning process. I-A Related Work The exploration of privacy-preserving techniques in the realm of machine learning has witnessed a surge in response to growing concerns regarding data privacy and security. Among these techniques, differential privacy (DP) has emerged as a very important framework, offering robust guarantees for privacy preservation. The work by [ 7 , 8 , 9 ] introduced deep learning with DP, paving the way for privacy-aware machine learning models. This work laid the foundation for incorporating DP into various machine learning paradigms, including meta-learning. In the context of meta-learning, Li et al. [ 10 ] explored DP meta-learning, emphasizing the need for privacy-preserving techniques in scenarios where meta-models are trained on sensitive or private datasets. Their work provides insights into how the challenges of few-shot learning and privacy concerns can be jointly addressed through the integration of DP. Zhou et al. [ 11 ] investigated the task-level DP approach within the meta-learning paradigm. By introducing privacy preservation at the task level, this work offers a nuanced perspective on securing sensitive information during the meta-learning process. The task-level granularity allows for fine-tuned control over privacy guarantees, addressing concerns related to both individual tasks and the overall meta-model. These advancements in incorporating DP into meta-learning underscore the ongoing efforts to reconcile the pursuit of improved machine learning performance with the imperative to safeguard user privacy in the face of increasing privacy challenges. I-B Motivations and Key Contributions From our observation, traditional implementation of privacy-preserving techniques in meta-learning like DP often leads to a drop in utility, impacting the performance of the models. To mitigate this issue, we propose an adaptive clipping technique. This technique aims to strike a balance between privacy protection and model performance by dynamically adjusting the clipping threshold during the meta-learning process. By doing so, sensitive information can be effectively protected while minimizing the adverse effects on utility, thereby facilitating more efficient and privacy-aware meta-learning algorithms. By applying our adaptive clipping technique to well-known meta-learning algorithms, such as Model-Agnostic Meta-Learning (MAML) [ 12 ] , Reptile [ 13 ] , and Meta-SGD [ 14 ] , we demonstrate its significant potential to improve their performance while ensuring robust privacy guarantees. This paper aims to investigate the effectiveness of our adaptive clipping technique in enhancing the utility of task-level DP meta-learning algorithms for few-shot learning tasks. We focus on popular meta-learning algorithms, such as Model-Agnostic Meta-Learning (MAML) [ 12 ] , Reptile [ 13 ] , and Meta-SGD [ 14 ] , and demonstrate how our technique can improve their performance with privacy guarantees. The contributions of this work are highlighted as follows: 1. We propose an adaptive clipping technique that dynamically adjusts the clipping threshold during the training of DP meta-learning algorithms. Our technique effectively balances privacy preservation and utility, allowing the meta-model to leverage available information while meeting privacy constraints. 2. We integrate the adaptive clipping technique into popular task-level DP meta-learning algorithms:Model-Agnostic Meta-Learning (MAML), Reptile, and Meta-SGD. Through extensive experiments on benchmark datasets, we demonstrate that our approach improves the performance and utility of these algorithms in few-shot learning scenarios. 3. We are the first to investigate the application of task-level DP to the Meta-SGD algorithm, a widely used meta-learning algorithm. By incorporating DP into Meta-SGD, we contribute to the development of privacy-preserving techniques in meta-learning and establish its feasibility and effectiveness in the few-shot learning context. 4. We provide a comprehensive convergence analysis of the adaptive clipping technique-enhanced meta-learning algorithms. Additionally, we conduct a thorough privacy analysis of our proposed techniques, quantifying the privacy guarantees provided by our DP meta-learning algorithms and analyzing the trade-offs between privacy and utility. 5. We conduct comprehensive experiments to evaluate the performance of our approach. We compare our adaptive clipping technique-enhanced meta-learning algorithms against other state-of-art approaches, showcasing significant improvements in both privacy preservation and predictive performance. I-C Paper Organization The remainder of this paper is organized as follows: In Section 2, we provide a brief overview of few-shot learning, its challenges, and the role of meta-learning in addressing these challenges. We then discuss the importance of privacy preservation in meta-learning and introduce DP as a privacy-preserving framework. Section 3