Latent Paraphrasing: Perturbation on Layers Improves Knowledge Injection in Language Models

Minki Kang, Sung Ju Hwang, Gibbeum Lee, Jaewoong Cho · 2024

Read Latent Paraphrasing: Perturbation on Layers Improves Knowledge Injection in Language Models by Minki Kang, Sung Ju Hwang, Gibbeum Lee, Jaewoong Cho (2024) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2411-00686.txt.

Description

Abstract As Large Language Models (LLMs) are increasingly deployed in specialized domains with continuously evolving knowledge, the need for timely and precise knowledge injection has become essential. Fine-tuning with paraphrased data is a common approach to enhance knowledge injection, yet it faces two significant challenges: high computational costs due to repetitive external model usage and limited sample diversity.

Catalog record

CREATORMinki Kang, Sung Ju Hwang, Gibbeum Lee, Jaewoong Cho
YEAR2024
COLLECTIONPAPERS
FORMATTXT
FILE SIZE60 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2411-00686

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Latent Paraphrasing: Perturbation on Layers Improves Knowledge Injection in Language Models Minki Kang, Sung Ju Hwang, Gibbeum Lee, Jaewoong Cho Abstract As Large Language Models (LLMs) are increasingly deployed in specialized domains with continuously evolving knowledge, the need for timely and precise knowledge injection has become essential. Fine-tuning with paraphrased data is a common approach to enhance knowledge injection, yet it faces two significant challenges: high computational costs due to repetitive external model usage and limited sample diversity. To this end, we introduce LaPael, a latent-level paraphrasing method that applies input-dependent noise to early LLM layers. This approach enables diverse and semantically consistent augmentations directly within the model. Furthermore, it eliminates the recurring costs of paraphrase generation for each knowledge update. Our extensive experiments on question-answering benchmarks demonstrate that LaPael improves knowledge injection over standard fine-tuning and existing noise-based approaches. Additionally, combining LaPael with data-level paraphrasing further enhances performance. 1 Introduction Pre-trained Large Language Models (LLMs) encode extensive factual information from their training data, enabling them to answer factoid questions such as “Who is the director of Dune: Part Two?” [ 4 , 32 ] . However, knowledge in LLMs is static, which can lead to outdated information as real-world knowledge evolves. Additionally, LLMs often lack specificity for specialized or private domains. To address this, it is common practice to fine-tune LLMs with updated or domain-specific documents, keeping the model’s knowledge up-to-date and enhancing expertise in particular domains [ 14 , 17 , 19 ] . Figure 1 : Effect of paraphrasing data in knowledge injection. However, does fine-tuning LLMs on a single document allow them to fully internalize its knowledge? Even in pre-training, Kandpal et al. [20] found that LLMs cannot perfectly learn all the information in the training data, particularly long-tail knowledge that appears rarely or only once. Existing work [ 33 ] has shown that this issue persists with fine-tuning and suggested that data augmentation, such as paraphrasing, is a simple yet effective way to enhance knowledge injection. As shown in Figure 1 , fine-tuning with paraphrases enhances knowledge injection, as evidenced by improved Question-Answering (QA) task performance. While data-augmented approach via paraphrasing is effective for knowledge learning, it has two main limitations: (1) High computational cost: Generating high-quality paraphrases requires significant computational resources. As shown in Figure 2 , paraphrasing models such as LLMs [ 5 , 7 , 11 , 58 ] need to repeatedly generate paraphrases for each document with the new incoming knowledge. This leads to higher costs as the number of documents being learned continually increases; and (2) Limited diversity in augmented data: Although LLMs can produce varying high-quality paraphrases by sampling from the generative distribution, the diversity of the generated text is limited, resulting in a narrow range of augmented samples at the discrete data level. One way to overcome these issues is to introduce noise into the token embedding. However, existing works [ 16 , 57 ] do not consider the text semantics when they perturb the latent features of LLMs with randomly generated noise. Figure 2 : A conceptual illustration of the proposed approach . On the left, we show the existing method of knowledge injection by paraphrasing each document for data-level augmentation. On the right, we present the conceptual illustration of LaPael with trained latent paraphrasers. Unlike the method on the left, LaPael can eliminate the need for users to repeatedly paraphrase using LLMs once latent paraphrasers are trained. To address these issues, we take a distinct approach using an input-dependent noise generator named “latent paraphraser” learned from the paraphrases. Specifically, this function perturbs early layers to augment LLMs at the latent level while preserving the meaning of the text. To optimize the latent paraphraser, we start by generating paraphrases of the documents. Then, we train the latent paraphrasers to ensure that the latent distribution of the LLMs with the original sentence is close to the latent distribution with the paraphrased sentences. Once training is done, we can transfer the latent paraphrasers to the documents from any domain that contains new knowledge. We refer to our method as La tent Pa raphrasing of Language Mod el s ( LaPael ), as it learns the paraphrasing of text data at the latent level. We validate our approach on diverse question-answering benchmark datasets [ 38 , 27 , 51 ] designed to evaluate knowledge injection. These benchmarks involve fine-tuning LLMs on documents that contain the knowledge required to answer the questions in the datasets. Our results show that LaPael significantly improves knowledge injection performance compared to standard fine-tuning. Moreover, LaPael outperforms fine-tuning with paraphrases, demonstrating that LaPael alone is sufficient for data augmentation in knowledge injection scenarios, as illustrated in Figure 2 . As shown in Figure 1 , we further find that using LaPael in combination with paraphrases further enhances performance, providing complementary benefits to data-level augmentations. Finally, LaPael surpasses existing noise baselines [ 16 , 57 ] , highlighting the importance of learning noise for effective augmentations. Our contributions are as follows: • We introduce LaPael , a new method that applies learned perturbations to the layers of LLMs to enhance knowledge injection, addressing the limitations of data augmentations and noise baselines. • We validate LaPael using diverse question-answering benchmark datasets, demonstrating a significant improvement in knowledge injection performance compared to standard fine-tuning. • Our results show that LaPael not only outperforms fine-tuning with paraphrases but also complements it, providing additional benefits when used together, surpassing the performance of existing latent noise-based methods. 2 Related Work Knowledge of Large Language Models Large Language Models (LLMs) store vast amounts of factual knowledge in their pre-trained parameters [ 36 , 44 ] . The straightforward way to extract the knowledge of LLMs is to ask the question that requires factual knowledge [ 43 , 58 ] . Through asking questions, Kandpal et al. [20] have found that LLMs cannot perfectly memorize the entire knowledge in the pre-training corpora, especially for knowledge that appears rarely or only once. To make LLMs answer the question requires under-represented or new knowledge, previous works have clustered into two different solutions. The first one is retrieval-augmented methods [ 26 , 39 , 42 ] that retrieve knowledge from an external knowledge base and input the retrieved knowledge alongside the question into LLMs. The second one is fine-tuning [ 12 , 17 ] where the parameters of pre-trained LLMs are continually updated by fine-tuned on the document containing knowledge in an unsupervised way as in pre-training [ 37 ] . In our work, we focus on improving the fine-tuning-based solution, as storing new knowledge in the parameters of LLMs is efficient since we can reduce the length of the input prompt and do not need any extra module or memory in the deployment time [ 6 ] . Knowledge Injection in LLMs In this work, knowledge injection in LLMs denotes fine-tuning LLMs on the set of documents to inject new or under-represented knowledge into LLMs [ 33 , 17 ] , different from another task of injecting symbolic knowledge (e.g., knowledge graph) into LLMs [ 55 , 54 ] . Among previous works, CaMeLS [ 14 ] has introduced a meta-learning method for learnable loss scaling

Read the complete text