FineGates: LLMs Finetuning with Compression using Stochastic Gates

Jonathan Svirsky, Yehonathan Refael, Ofir Lindenbaum · 2024

Read FineGates: LLMs Finetuning with Compression using Stochastic Gates by Jonathan Svirsky, Yehonathan Refael, Ofir Lindenbaum (2024) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2412-12951.txt.

Introduction

Jonathan Svirsky, Yehonathan Refael, and Ofir Lindenbaum tune large language models with stochastic gates in a 2024 paper. Full tuning of a huge model costs much and can overfit small data. Light adapters help, but they leave the base model's bulk untouched.

Their adapter learns gates that switch parts of the frozen model on or off per task. Sparsity and task fit are learned together, with few new settings. The gates speed up the base model at run time while keeping accuracy. Variants with low-rank add-ons were also tested against recent rivals. The method held its own on accuracy and let twenty to forty percent of the model be dropped with little loss.

Catalog record

CREATORJonathan Svirsky, Yehonathan Refael, Ofir Lindenbaum
YEAR2024
COLLECTIONPAPERS
FORMATTXT
FILE SIZE39 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2412-12951

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

FineGates: LLMs Finetuning with Compression using Stochastic Gates Jonathan Svirsky, Yehonathan Refael, Ofir Lindenbaum Abstract Large Language Models (LLMs), with billions of parameters, present significant challenges for full finetuning due to the high computational demands, memory requirements, and impracticality of many real-world applications. When faced with limited computational resources or small datasets, updating all model parameters can often result in overfitting. To address this, lightweight finetuning techniques have been proposed, like learning low-rank adapter layers. These methods aim to train only a few additional parameters combined with the base model, which remains frozen, reducing resource usage and mitigating overfitting risks. In this work, we propose an adaptor model based on stochastic gates that simultaneously sparsify the frozen base model with task-specific adaptation. Our method comes with a small number of trainable parameters and allows us to speed up the base model inference with competitive accuracy. We evaluate it in additional variants by equipping it with additional low-rank parameters and comparing it to several recent baselines. Our results show that the proposed method improves the finetuned model accuracy comparatively to the several baselines and allows the removal of up to 20-40% without significant accuracy loss. 1 Introduction Large language models (LLMs) have revolutionized natural language processing by enabling powerful and versatile applications across various tasks. These models, pre-trained on vast amounts of text data, possess a deep understanding of language, making them valuable for text generation, translation, and sentiment analysis tasks. However, finetuning is often necessary to tailor these models to specific applications. Finetuning allows the model to adapt to the nuances of a particular task by updating its parameters based on a smaller, task-specific dataset. The challenge is that users typically have limited data for finetuning, which can constrain the model’s ability to optimize for the desired task fully and may lead to overfitting or suboptimal performance. Despite these challenges, finetuning remains crucial in leveraging the full potential of large language models for specialized applications. Recently, several innovative methods have been proposed to optimize the finetuning process of large language models, addressing the challenges associated with limited data and the computational cost of updating all model parameters. One such approach is LoRA (Low-Rank Adaptation) ( Hu et al., 2021 ) , which introduces a more efficient way to finetune models by freezing the original parameters and injecting trainable, low-rank matrices into each layer of the model. This technique significantly reduces the number of parameters that need to be updated during finetuning, making the process both faster and less resource-intensive. Most recent efforts in optimizing finetuning ( Zhang et al., 2023a ; Chavan et al., 2023 ; Xu et al., 2023 ; Li et al., 2022 ; Lin et al., 2024a ; Bałazy et al., 2024 ; Hu et al., 2021 ) , including LoRA, focus on adding new parameters while keeping the base model frozen, ensuring that the original pre-trained knowledge is retained ( Rozner et al., 2024 ) . When training models using low-rank adapters ( Kopiczko et al., 2023 ; Zhang et al., 2023a ; Lin et al., 2024a ; Bałazy et al., 2024 ; Hu et al., 2021 ) , the number of updated parameters is reduced, resulting in faster convergence during finetuning. However, the inference runtime remains the same as it still depends on the base model size. To treat this issue several methods were proposed large language models (LLM) compression like like quantization and pruning. Quantization reduces the memory usage of language models by converting their parameters into lower-bit data types ( Bondarenko et al., 2024 ; Lin et al., 2024b ) . Although quantization reduces the memory consumption of language models, its speedup advantages rely on specialized framework support, which limits its flexibility and adaptability. Pruning ( Ma et al., 2023 ) aims to improve inference efficiency in language models by removing unimportant parameters. Structured pruning ( Xia et al., 2022 ) removes consistent blocks of parameters, or model dimensions, achieving more general inference efficiency improvements. This method often involves knowledge distillation ( Hinton, 2015 ) , increasing training costs. In this work, we propose a simple, efficient, and effective adaptation method that adopts the base model for the target task by learning stochastic gates on the weights of the base model with optional low-rank updates. Instead of only adding low-rank matrices for adaptation, we train gates to preserve only task-specific information within the base model itself. This approach allows us to maintain the integrity of the pre-trained model while incorporating the essential nuances of each specific task directly into its structure, leading to a more effective finetuning process. Our method not only preserves task-specific information within the base model but also allows for a significant reduction in the adopted base model’s layer weights. By efficiently embedding the necessary task-specific details while reducing unnecessary parameters, our method enhances both the effective performance and efficiency of the finetuned model, making it better suited for real-world applications where speed and resource usage are critical. In addition, our model is optimized in an end-to-end fashion without requiring post-training pruning, incorporating optimization sub-stages during the training, or increasing the overall finetuning time. We evaluate our method on Transformer-base models and show that effective finetuning is achievable along with base model parameters compression, which could be reduced by up to 40\% without significant loss in accuracy. In the next sections, we present our method and the empirical evidence of its effectiveness. We also provide a convergence proof of our method. 2 Related Work 2.1 Low-Rank Adaptation Low-rank adaptation aims to tune LLMs with limited resources by updating a small number of parameters by tuning injected layer modules ( Pfeiffer et al., 2020 ; Houlsby et al., 2019 ) , embeddings ( Lester et al., 2021 ; Li and Liang, 2021 ) or training with a low-rank structure of the gradients ( Refael et al., 2024 ) . One widely used method, LoRA ( Hu et al., 2021 ) , tunes low-rank decomposed layers to avoid training cost overhead. However, LoRA keeps the tuning layer shapes in the base model static without dynamic adjustments. Another work by ( He et al., 2022 ) dynamically adjusts tuning parameters during training, and ( Zhang et al., 2023b ) gradually reduces tuning parameters, but neither of them benefits the inference efficiency of the finetuned model. 2.2 Finetuning with Pruning There are two main types of model pruning during finetuning: structured ( Xia et al., 2022 ; Zhao et al., 2024 ) and unstructured ( Sanh et al., 2020 ) . Unstructured pruning prunes the most unimportant parameters in the model without any order, while structured pruning prunes entire blocks, rows, or columns in the weight matrices. Moreover, a post-training pruning method, i.e., proposed by ( Frantar and Alistarh, 2023 ) , aims to prune finetuned models with limited extra costs but requires initialization from fully finetuned models. Compared to these methods, our approach focuses on accurately adapting the model to the target task while pruning 20-30% of the base model parameters, with minimal accuracy loss. Additionally, the pruning process is integrated with model adaptation, ensuring no extra training time is required. 2.3 Finetuning with Adaptive Pruning A recent work that closely aligns with

Read the complete text